Are business schools intellectually bankrupt?

From the New York Times today, a report from business school professors concerning a study which claims to show that professors are prejudiced, too. I remember reading the original paper on this study, which made it painfully clear that the authors were pursuing an agenda and that they arrived at their conclusions by scouring their data for correlations which supported their case, a classic hallmark of poor science. But perhaps sound methodology is too much to expect from business school professors?

The reason I paid particular attention to this study was that I was one of the participants. Here was the original email I received.

Dear Professor XXX,

I am writing you because I am a prospective doctoral student with considerable interest in your research. My plan is to apply to doctoral programs this coming fall, and I am eager to learn as much as I can about research opportunities in the meantime.

I will be on campus next Monday, and although I know it is short notice, I was wondering if you might have 10 minutes when you would be willing to meet with me to briefly talk about your work and any possible opportunities for me to get involved in your research. Any time that would be convenient for you would be fine with me, as meeting with you is my first priority during this campus visit.

Thank you in advance for your consideration.

I remember receiving this email. What immediately struck me was the repeated vague references to “my research.” Now in order to have any appreciation of my research, you would, at the very least, have to know that that Langlands program exists, or that my research is related to Wiles’ proof of Fermat’s Last Theorem and algebraic number theory. The fact that there is no mention of number theory nor any indication of the background of the student immediately links the email (in my mind) to academic spam. Surely it’s the case that my reaction would be shared by many academics? Who is so desperate for attention that they would imagine this email reflects a genuine personal interest in their work? As you would expect, I completely ignored the email and promptly forgot about it. Then, a week later, I received the following email:

Dear Professor XXX,

Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you received this message. We hope you will see the value of the knowledge we anticipate producing with this large academic study.

We are decision-making researchers interested in how choices differ when they are made for “now” versus for “later”. Previous research has shown that people tend to favor doing things they viscerally want to do over what they believe they should do when making decisions for now, while they are more likely to do what they believe they should when making decisions for later (for a review, see Milkman, Rogers and Bazerman, 2008). The email you received from a student asked for a meeting with you either today (if you were randomly assigned to the “now” condition) or in a week (if you were randomly assigned to the “later” condition). This email was actually from a fictional student. It was designed for a study of the responsiveness of University faculty to meeting requests from prospective students of various backgrounds made on short notice versus well in advance. Faculty members at the top 260 U.S. Universities (as ranked by U.S. News and World Report) and affiliated with Ph.D. programs were identified as potential participants in this study, and a random sample (6,300 faculty in total – one per Ph.D. program) were selected to receive emails. In addition to examining the responsiveness of faculty to meeting requests for “now” versus “later”, we are also interested in how the identity of the applicant affects, or does not affect, response rates, and as such, the name of the student sending a meeting request was varied (by race and by gender). We expected that students from underrepresented groups would receive fewer meeting acceptances than other students, though we have competing hypotheses about whether this would effect would be stronger in the “now” or the “later” condition.

I love the line concerning the fact they the have “competing hypotheses about whether this would effect would be stronger in the “now” or the “later” condition” — see, they prove their case whatever the data!

Given the ridiculousness of the initial email, I was appalled that my response might contribute to some published data implying that professors were dismissive of minorities and/or women. Let me be clear at this point that it may well be the case that academics are more dismissive towards women (I fear that this may indeed be true in our field), but I am convinced that this study would have little of academic value to say on the matter. That said, having received this second email, I did nervously look back to my initial email to see what fake name I had failed to respond to.

The answer: Steven Smith. Yes! I ignored the name that could easily be a white male.

Update: For a different take on this study by Andrew Gelman (who, apparently, is more willing to spend time answering random emails than I am) see here, here, and here.

This entry was posted in Politics, Rant and tagged , , , . Bookmark the permalink.

10 Responses to Are business schools intellectually bankrupt?

  1. Would you care to elaborate on the “poor science” aspects?

    • I was merely referring to the fact that, if you compute enough random independent variables \{X_i\}, you will find correlations between them.

      My first main intellectual complaint concerns the fact that they have “competing hypotheses” concerning whether the effect of gender/race was more pronounced for emails seeking access in the distant future or the near future. My understanding of the scientific method is that you can’t have a hypothesis for either conclusion and declare success regardless of the outcome. My second complaint is, to repeat myself, the poorly formulated initial email.

    • In fact, things may be even worse than I first thought. In the email above, the claim is made that

      Previous research has shown that people tend to favor doing things they viscerally want to do over what they believe they should do when making decisions for now, while they are more likely to do what they believe they should when making decisions for later

      Which suggests that they actually expected that women/minorities would be worse off in immediate term rather than the longer term. Yet, looking at the data, the positive response rate for an immediate meeting was higher for the following groups:

      Caucasian Females, African American Females, African American Males, Hispanic Females, Indian Females, Chinese Males

      than it was for caucasian males (with relative differences 5%, 8%, 0%, 1%, 1%, 4%, and 1% respectively) and only lower for Indian males and Chinese Females (-8% and -4%).

  2. KV says:

    I was under the impression that the “visceral” vs. “should” was with respect to meeting with the student at all. You viscerally don’t want to bother meeting with some kid who emails you out of the blue, but you feel like you probably should. Then on top of this study, there was the totally separate study of whether response rates varied between ethnicities/genders.

    Also, the NYT article you linked contains the following quote:

    “Professors were more responsive to white male students than to female, black, Hispanic, Indian or Chinese students in almost every discipline and across all types of universities. ”

    This appears to be blatantly false given the numbers you just cited.

  3. Also, what is the difference between the first and the second paper? (I tried reading the second paper, but it is verbose beyond belief, and getting to the point requires a longer attention span than I seem to possess…)

    • Hmmm, it seems that perhaps there is only one paper (from 2012). Somehow, I inferred from the fact that the NYT opinion was just written that there was a follow up paper on the same data, but I think I was mistaken. (No, I was mistaken that I was mistaken, there is a second paper.)

  4. NYT does actually mention two papers…

  5. I think there were several versions of the paper which was initially published some time before 2010.
    It’s a fabulous example of junk science marketing.
    It looks like the researchers spent their grant money looking for evidence of discrimination, then when the results were random, they looked for whatever way they could to make it fit a narrative. Then when that narrative didn’t draw media attention, they revised it and revised it and revised it until, voila, they penetrated the NY Times/NPR/Guardian/blogosphere narrative hive.

  6. Pingback: Public Displays of Mathematics | Persiflage

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s