When should you notify an applicant of the poor quality of a reference letter?

January 10, 2012 4:04 PM

Academic ethics question: under what circumstances should you notify an applicant of the poor quality of a reference letter?

You are a reviewer for a granting agency that awards postdoctoral fellowships. The PhD supervisor of Candidate X has provided a reference letter which, while very positive, is exceptionally shoddy and poorly written - e.g., serious grammatical errors in almost every sentence and atrocious writing in general. While a few typos are to be expected in any reference letter, this one is of such terrible quality that it is difficult to read and reflects negatively on the author, and thus on Candidate X. (You have no reason to believe that this is an ESL problem).

You have the option to provide comments to Candidate X. Do you mention the poor quality of the reference? If so, how?

posted by googly to Education (24 answers total)

If it's the PhD supervisor, the candidate cannot avoid getting letters from them. And I can't imagine any way of the candidate confronting the advisor about it either. So while I guess you could mention it, I don't see how it would help things...
posted by redlines at 4:09 PM on January 10, 2012 [3 favorites]

Any reason to think the letter was written by an assistant and not proofed by the signer? Or that the writer has dementia or a substance problem?
posted by Lesser Shrew at 4:14 PM on January 10, 2012 [1 favorite]

What could the candidate possibly do about it? Don't mention it to him.
posted by ThePinkSuperhero at 4:16 PM on January 10, 2012 [7 favorites]

Seconding LS: I have first-hand experience knowing that professors (especially the more senior/busy among them) often don't write their own recommendation letters. A subordinate has likely either written or proofed it.

I say tell them and let Candidate X sort it out-- he will know who wrote it and will be able to give feedback to his supervisor. At the very least, he will know; when it comes to the overall quality of the whole application (including the reference letters) for things like fellowships and grants, ignorance is not bliss.
posted by supercres at 4:17 PM on January 10, 2012 [6 favorites]

I would tell him if he were my friend or if I had interacted with him more than one or twice professionally, but that seems like a step to far absent that. If the other letters are fine, so that most people will just ignore the silly one, this would lessen the need to intervene.

As to what he can do about it, he can find a different letter writer for future applications. Agreed it's more of a problem if it's his advisor.
posted by deadweightloss at 4:20 PM on January 10, 2012

It's also not unheard of for supervisors with students in very good standing to ask candidates to write their own letters and submit them for signing,or at least to provide the meat of the letter highlighting the qualifications and fit with program being applied to - so this is another vote for telling the candidate. If it's noticeably different from his or her own writing, maybe it's not his or her work, but there's an outside chance that it is.
posted by Miko at 4:21 PM on January 10, 2012

If you do this, and I'm not saying you should, you should definitely do it by telephone. Do not do it in writing.
posted by R. Schlock at 4:21 PM on January 10, 2012 [2 favorites]

Response by poster: Any reason to think the letter was written by an assistant and not proofed by the signer? Or that the writer has dementia or a substance problem?

Entirely possible, but impossible to know.
posted by googly at 4:21 PM on January 10, 2012

Is it at all possible to contact the letter writer and say something along the lines of, "If you honestly want to help this candidate, you need to re-write this" rather than contacting the candidate? I just hate the thought that this poor applicant is missing out on opportunities because his letter writer is lazy and/or careless.
posted by synecdoche at 5:00 PM on January 10, 2012 [3 favorites]

At which point in the process would you be giving this feedback? Would Candidate X be able to submit a new letter to be considered in this application cycle, or would this be part of the feedback from the reviewers that the candidates get back after finding out whether their application has been accepted? I work for a granting agency that awards postdoctoral fellowships, and what I would suggest is that if there is a possibility that a new letter could be submitted, you alert the administrative staff and have them take care of it. At my agency, I would write the candidate and gently suggest that perhaps the letter writer submitted a draft of his letter instead of the final copy and offer the opportunity to send a new one.

If instead you're simply offering reviewer feedback that the applicant gets so he can revise future applications, I would point out that the quality of the letter impacted his scoring - that seems like information he should have for the future.
posted by Neely O'Hara at 5:06 PM on January 10, 2012 [1 favorite]

that seems like information he should have for the future

Absolutely; my thoughts exactly. If you're giving him feedback about why his application was denied, it would be unethical not to tell him this. What he does with it is his business, and is between him, his PI, and his PI's ghostwriter, if applicable.

I'd tell him even if you accepted the application.
posted by supercres at 5:49 PM on January 10, 2012 [3 favorites]

This happened to me, although the circumstances were different (the letter in question was actually someone else's letter with my name substituted for theirs, so there were serious discrepancies in details, to say the least, between the materials I submitted and this letter. Also, I already had a good personal/working relationship with the people reading the application materials, so the discrepancies stood out to them even more). Knowing about it gave me the chance to figure out what happened, and to be careful using this person as a reference from now on.

I am very grateful to the people who let me know I had a bum reference letter, so I think you should let the applicant know what's up. They may be relying on this person's references for a while, and it's only fair to let them know that this might not be a good idea.
posted by heurtebise at 6:04 PM on January 10, 2012

I once had a letter of "recommendation" from a boss/advisor that apparently was not flattering. After I applied, my secondary supervisor (who had this same boss) came up to me one day and said that he had been asked by the committee to supply a supplemental letter of recommendation. I did not talk with him at great length about how this boss should have been too professional to hold a previous disagreement against me, but that seems to have been what happened; the recommendation talked about my talents and how (much better) suited they were for a job that was nothing like the career path I was pursuing. They had asked him whether he agreed with this assessment, and offered him the chance to provide a letter that would substitute for the first one in the file.
The extenuating circumstances were, this was for a permanent job posting within the organization that she worked for (and I therefore contracted for). The people hiring knew that person, and while they didn't know me directly, knew a lot of other people I'd worked with, probably asked others for the dirt on the situation, and knew exactly who to go to when asking for a new letter.
(as it happens I didn't get the job but that's because I flubbed the interview.)
posted by aimedwander at 6:29 PM on January 10, 2012

Aside from contacting the letter writer directly, which I think you should do, couldn't you contact the director of graduate studies in the candidate's department? Or is it the case that the candidate is no longer a graduate student?
posted by Jonathan Livengood at 7:51 PM on January 10, 2012

What could the candidate possibly do about it?

He could get someone else to write a reference letter and drop the poor-writer reference. This seems obvious; did I misread the situation?
posted by zardoz at 7:57 PM on January 10, 2012

zardoz, I assume it would look bad for Candidate X not to have, among the referrals, a letter from the PhD supervisor because it would seem X was not on good terms with the supervisor and had gone around that person when asking for letters of recommendation.

I'd really want to contact the PhD supervisor about it, honestly. I think I'd lean towards saying that you received a letter for candidate X that had, perhaps, been intended to be a first draft, and you wanted to give supervisor a heads-up about that, as it appeared to have been sent along before the supervisor had a chance to proof it and make corrections. Give the person a little room to save face that way, and help Candidate X out (hopefully). If that didn't result in a better letter, tell X.
posted by misha at 8:45 PM on January 10, 2012 [12 favorites]

zardoz, if it were almost any other kind of application, an applicant could find a substitute letter writer, but I can't fathom a postdoc getting funding without a letter from their doctoral advisor - I would think the lack of such a letter would be more detrimental than even a poorly written letter.
posted by Neely O'Hara at 8:45 PM on January 10, 2012

Oh, I really like misha's idea. Addresses the issue directly without causing undue embarrassment for the PhD supervisor. Tactful, and generous of you.
posted by amicamentis at 12:07 AM on January 11, 2012

Maybe I come from somewhere completely weird but I had three PhD supervisors, all equally able to write letters of recommendation for me (two or three is standard at my University in NZ). Even the place I now work (in Ireland) where one supervisor is normal, students pretty much always have a secondary advisor which also knows their work very well and would make an appropriate reference. So hell yeah, I'd want to know.

But then I also had the kind of relationship with all my supervisors where I could have asked them about a badly written letter of reference and discussed getting it fixed up with them, particularly if it was a good reference but just poorly written. I did actually talk to them about what kind of letters they would write and which ones I should choose for which job (most jobs wanted two references so I had to leave one supervisor out). It was never a big deal and just part of the normal PhD finishing process. Not all PhD supervisors are some ego-ridden power-hungry person standing off in the distance ready to screw people over at the smallest slight, most of them actually care about helping their students get good jobs.

So definitely tell this student along with whatever other feedback you are giving about the applications. Grant feedback is really important to us even when the grant isn't funded. Don't let them keep making the same mistake when there are so many ways it can be fixed. Getting a postdoc is hard enough without this kind of stuff holding someone back.
posted by shelleycat at 12:25 AM on January 11, 2012

I have first-hand experience knowing that professors (especially the more senior/busy among them)

This and similar comments are wholly false in my 20 year experience in academia. I am a busy senior academic, and so are most of my friends and closest colleagues. None of us would even *consider* farming out the actual drafting of a letter of recommendation for a PhD advisee applying for jobs and postdocs. Few of us have "assistants" (as so many people think, and Law and Order seems to think we have 3 or 4 and that the entire department office staff exists to take care of routine business so the profs can think Big Thoughts). This is total bullshit. I have had "assistants" (usually working for me under some sort of grant funding) print out and prepare letters for my signature based on my original base letter (they might do something like change the salutation or the name of the job the candidate is applying for, but never any substantive text) but mostly -- even as a *department chair at a top 10 university* -- I write my own letters, print them out, proofread them, sign them personally, put them in envelopes, apply a stamp, address the envelope and mail them myself, both because there's really no one else who can (my grants don't cover things that would just make my life easier) and because it's broadly speaking an ethical obligation to do this stuff, and to write serious, thoughtful, well constructed, detailed letters for our PhD advisees *above all*!!!! After all, my own reputation as a PhD adviser *depends* fundamentally on my students getting postdocs and jobs.

Any professor who has someone else write his LoRs for PhD advisees (or in my view, anyone else they write for) is committing professional malpractice of a very serious sort. I can assure you it is not routine, at least not in the humanities and social sciences. The most famous rockstar-level colleagues I know (and that's many given where I work) wouldn't think of having anyone else draft the base letter of recommendation even for an undergraduate applying for grad school. Nor would any of us *ever* ask a PhD student to "write her own letter" for us to sign for something like a postdoc (or job) application. We do that for stupid things like immigration procedures etc, and even then we proofread and change the letter substantially (I just did this with a green card application support letter -- let the student write it with his lawyer, then heavily edited it to make it sound like my voice and tell the truth precisely as I see it).

Academia is different from business in this respect. A CEO probably doesn't write her own letters of recommendation for staffers. But any professor who doesn't do so for students -- not just, but especially doctoral advisees -- is a disgrace to the profession.

That said, you have no recourse in this situation, and advising anyone of the content of that letter who is outside the specific chain of the particular search process your in is itself an ethical violation.

I have had friends at other universities give me very helpful tips on changes I could make to advisee LoRs *after* searches on which they had served had closed, and in very broad terms (tone down the praise, say more about the post-doctoral project and less about the dissertation, etc). But beyond that, this student chose a shitty adviser and the chances are that if the adviser is illiterate or hates the student, the dissertation sucks too.
posted by spitbull at 5:47 AM on January 11, 2012 [1 favorite]

particular search process your you're in is itself an ethical violation.

See, profs can make typos too.
posted by spitbull at 5:49 AM on January 11, 2012

I kept thinking about this. If you are yourself a professional colleague of the prof who wrote the crappy letter, and if you know him well enough to do so, you could say something very discreet (and in person only, like at a conference) to *him* about how his letter sunk a candidate. Of course if he's far senior to you, you probably shouldn't bother doing that.

The key point is that it is in a professor's very best personal interest to only write for students about whom we feel we can say positive things (and make fair, balanced criticism, which improves even the very strongest letter in many cases, making it more believable) and then to write a well-crafted, succinct but detailed, and carefully proofread letter for our PhD advisees (or any student, but it's utterly unforgivable not to do this for your PhD advisees). Not only does their success on the job market constitute *the single most important* metric of your quality as a graduate adviser and more generally as a scholar, but your reputation for honesty and clarity -- your own personal scholarly integrity and standing -- will suffer if you write sloppy, overinflated, thoughtlessly negative, or generic letters for your advisees, over time and for sure. Of course, you start the process of screwing yourself professionally by accepting PhD advisees in whom you don't believe strongly.


So this guy is damaging himself as much as his advisee. Of course, he already has a tenured position and his advisee is having her/his career royally fucked. A bad letter from your adviser -- either negative or sloppy -- is a clear sign to a search committee that you did grad school wrong.
posted by spitbull at 7:37 AM on January 11, 2012

Response by poster: Thanks for all the very helpful insights. To address a few issues that have come up:

(1) There is no way to have the referee submit an updated reference letter - deadlines are already long past.

(2) I have no professional or personal connection whatsoever to the author. Direct contact would mean that the author would receive an email/phone call from a total stranger in an unrelated field questioning the quality of their reference letter. I am not willing to do this.

(3) The feedback to the candidate would be part of an anonymous reviewer's report sent by the funding agency to the candidate.
posted by googly at 10:40 AM on January 11, 2012

I would put something very delicately phrased in the feedback. (By "delicate" I mean institutional cowardly soft-peddling.)
posted by Lesser Shrew at 1:07 PM on January 11, 2012