Letters of Recommendation Should Not Be a Component of Residency Applications

Part I

What is valued in medical education and residency selection has changed enormously since I was a student. We used to value things that were easy to measure even if they had only a tenuous relationship to the ability to doctor. Now, we seem to be unable, or unwilling, to assess what we now value in students. Dr. Van Cleve writes about why letters of recommendation are not helpful in assessing residency applicants.

This piece is a bit longer than our usual but I have not been able to stop thinking about and talking about it since I first read it. We divided it into two parts. Part I describes the problems with letters of recommendation and why they are not useful. Part II closes the argument about why we should abandon them.

Adam Cifu

In the television series Colin from Accounts, Harriet Dyer plays Ashley, a final year medical student in Australia who is trying to figure out life, love, and how to care for a disabled dog. We watch Ashley as she sees patients, repairs perineal lacerations, and generally seems to live the harried life of a mid-level US medical resident. Her relative independence as an Australian medical student, however “juiced” for comedy, has made me reflective about the experiences of 4^th year of medical students here in the US, especially as I’ve been holding advisory meetings with 4th year medical students who are preparing applications to train in my specialty, anesthesiology.

Each student sends me an email with his or her board scores, clerkship grades, and a personal statement. We meet, I offer feedback, and we strategize on which programs are “realistic” for them. I gently ask about their contingency plans should the increasingly competitive Match fail to yield them a position in anesthesiology.

What the students can’t send me to review are their letters of recommendation. Letters of recommendation, written by physicians who have supervised the student, are a required component of the application package for residency. Students are asked to waive their right to review these assessments under the assumption that confidentiality will make the letter writer more honest. Each student is expected to find at least three faculty members to write in support of their application.

While I increasingly find the residency application process grim when it should be exciting, it is the inclusion of letters of recommendation that truly drives me bananas.

Consider this: before I started medical school, I decided I wanted to spend the summer cycling across America, starting in my home state of Virginia and ending in Oregon. I planned to camp each night, and I learned from books (actual printed books) that there were churches and community centers that would let traveling cyclists camp for free if there were no nearby campgrounds. My mother told me that I should go to our parish priest and ask for a letter, “on church letterhead”, that introduced me as a good Christian of upstanding character. If someone hesitated to let me set up my tent next to the parish hall, I could take out the letter and all would be well.

Now, 23 years later, I still remember the strange look on one church warden's face when I offered to dig this letter out of my saddlebags. It’s the same look I think that residencies should be giving the entire concept of letters of recommendation. Who in their right mind would trust a stranger’s opinion about another stranger?

Writer Bias

While on clinical clerkships, medical students must find supervising physicians willing to write them a letter. When I was a student, I asked for a letter from a doctor who I so idolized that my girlfriend (legitimately) asked whether I’d fallen in love with her. This mentor agreed and told me that she always showed her letters to their recipients, proudly stating that “waiving your right to see the letter doesn’t waive my right to show it to you.” The letter was so effusive in its praise for me that I asked her if I could send a copy to my mother as a Christmas present.

I was a good medical student, but I wasn’t by any means the best at my school. I simply got lucky and asked the right person for a letter. There was no way I could have known that she treated letters with such care and attention. If the point of a letter of recommendation is to help the reader predict something meaningful about the subject, of what possible use is a process so heavily dependent on the attention and writing skills of the author? Are medical students supposed to learn how to determine which faculty members will pour out their unsatisfied literary ambitions in service of their student’s future?

Residency programs have known for years that writers can’t be trusted to be consistent in their evaluations. Some fields now require “standardized” letters of recommendation. These reduce the letter from a flowery Christmas present for a doting mother to a form that confines the interaction between evaluator and student to specific dimensions. How long did the letter writer work with the student? With what frequency? In what quintile of performance would they rank the student's work? This piecemeal approach appears to solve the inevitable problem of writer bias, but it also serves to make every letter look the same, making the problem of sorting medical students even harder.

Tuition dollars must become residency positions

“But surely these writers aren’t strangers,” you might say, suggesting that medicine remains a small and exclusive club, and that a letter from someone I know, or at least whose name I recognize, will carry weight with me as a reader. The student gets the opportunity to demonstrate a little knowledge of the politics of the field by finding a prominent person willing to lend a little of their reputation to the student’s cause.

And there is something to this idea: if a good friend or former colleague called me up and told me that a student was the “real deal,” I have to say, given the relative lack of objective data in the process, that I’d take that seriously. But when you expand that circle of reputation and connection just a little farther, it collapses. Not infrequently, we admit a resident with personality issues so deeply problematic that they create chaos from the moment of their arrival. These physicians nevertheless seem to have found beautiful letters written by prominent doctors.

How does this happen?

First, medical students know that one of their jobs is to be obsequious to the point of absurdity. I’m embarrassed to tell you that when I was a student, I used to bake my team muffins for breakfast, and I’m not well known for being particularly attuned to what people think of me. A letter that describes someone who poured weeks of deference into a make-or-break rotation is about as honest and useful as a Hallmark card.

Second, the role of the medical student has been continually diminished by an increased concern for patient safety, billing compliance, and worries about student wellbeing. Students now only rarely have the opportunity to demonstrate (or fail to demonstrate) the characteristics they’ll need to exercise as physicians.

Third, and perhaps most importantly, medical schools realize that they must find jobs for their problematic students. Schools used to believe that one of their functions was to identify people who wanted to be doctors, managed to get into medical school, but then demonstrated somehow that they should absolutely not be doctors. Based on my experience with several such individuals, this is no longer a priority. Medical schools have evolved, like many other gateways to membership in the elite, into a credentialing system that turns tuition dollars into residency positions. Virtually every medical student whose loans get funded will find a letter that indicates that they richly deserve a residency position, because a medical school cannot afford to be seen as a place where the process of credentialing fails to occur.

Reader Bias

Until very recently, I would read hundreds of letters each year. Around the 50th letter, they curdle and congeal into a platitudinous mush.

“The best student I can recall in my 20 years in practice.”
"A true star"
“Let me tell you, this student has ‘it’.”
“I unreservedly give this student my strongest recommendation.”

What am I to make of these accolades? More importantly, how does what I make of it compare to what every other faculty member who reads the same letter, each with their own priorities and interests, make of it? When I come across a letter that truly stands out, one that perhaps reminds me of the letter I gave to my mother, I pause, and then I ask myself whether I have any idea whether I’m more fond of the student or the writer.

Over the past few years, like almost every other academic physician in America, I’ve been asked to learn about, accept, and attempt to mitigate my own implicit biases. In the context of applications and interviews, I’ve been taught that I tend to like people who look like me, who share my interests, who are tall, well groomed, and speak with (and without) certain accents. The entire process of reviewing a letter of recommendation strikes me as a designed exercise in implicit bias. How much do I like the student having read what a stranger thinks of them? How can I possibly explain my assessment of this nakedly biased piece of writing as anything other than an integral of my own personal priorities?

What I should do, I think, is evaluate the student against a set of institutional priorities for the type of person who should become an anesthesiologist. In the next section of this piece, I will describe why that has become increasingly difficult.

Wil Van Cleve MD MPH is a practicing anesthesiologist in the Pacific Northwest. The views expressed here reflect his personal opinions and not those of his employer.