Well, hardly anyone noticed Gary Sheffield's recent charges of racism.

Re: Hewitt. I'm not a big tennis fan (mostly majors and a few Masters events), but I'm pretty sure Hewitt was universally considered the biggest d-bag on the Tour in his prime. His sudden decline has made no one sad.

Someone else (in a thread some weeks ago) said this on Primer: the home plate ump should announce the ball and strike calls, but those calls should be determined by a Questec-like technology (one that tells him if the pitch was a strike or not). As far as I know, that can be implemented now and with far less inaccuracy than with an umpire making the decision. I like this suggestion.

I thought someone familiar with the questec system recently posted here that questec required a lot of post-processing; that is, the vertical portion of the strike zone was calibrated after the game, based on the video. Apparently they don't do it in real time.

This also brings up another question. The questec operator responsible for setting the vertical strike zone would also be susceptible to racial influence. And if it is done via post-processing, issues like attendance and count may have less of an impact.

Well on one hand you have Clayton Bigsby and on the other hand you have 3rdBase. Which path do I follow?

One finding that casts considerable doubt on the racial bias conclusion is that the hitter's race had no impact. The hitter is about 24 inches from the ump, vs. 60 feet, and not infrequently speaks to the ump and/or catcher. Hitters are also much more likely than pitchers to express disagreement with prior pitch calls, audibly or through facial expression, in a way the ump could detect. So the ump's personal feelings about the hitter -- whether based on race or other factors -- have to be operating at least as strongly, probably much more so, than the ump's feelings about the pitcher.

I haven't read the study, but do they make a distinction between black Africans and black Caribbeans? I ask because an umpire, who either is consciously or uncounsciously racist, there shouldn't be a difference.

Also I recall a study done on the uniform color of football teams, where the team with the dark uniform would get more penalties than the lighter team color irrespective of the race of the players. Did they also try to look at uniform color to verify its impact?

If you disagree with the study, that's fine - I disagree with studies all the time. But the Wolfers-Price one, I think, is pretty well done with the limitations they had. Economists have a hell of a hard time getting data....

No doubt there. I don't disagree with the study's finding that the refs have a racial bias. They may, they may not, I'd lean toward that they do. They're human, people favor what is more familiar to them, and for most people that is other people of the same race, religion, education, etc.

What I've been arguing against in this thread is poor science. Have I been bashing the NBA ref study because I'm a closet racist? No. Like I said earlier, I'm a little biased, I'll admit it. As others have noted, bias does not imply racism. The poor science comes in in that the NBA study was poorly done, in that a crew of 2 white refs + 1 black ref = 3 white refs or 2 black refs + 1 white ref = 3 black refs. Unless you can convince me that the white refs exert that much influence over the black ref, or the black refs over the white ones, I don't believe that. I think that is poor data collection and methodology, and all claims made off that data are just as poor spoiling the study. Yet, that doesn't stop the authors and Wharton from going to the NY Times, the WSJ, ESPN, ABC, CNN, etc. Poor science to gain publicity.

Again, while I haven't read the MLB study, if what everyone is saying here is true, that at max it is one pitch a game, then you have to compare this to the "normal" errors in umpiring. Even then, going to Time magazine and having an article written up seems to me self-publicity rather than science. Most people aren't going to read the whole thing, or they may not even read the study (hey, I haven't yet), yet they'll see the headlines and all the sudden Schrutebag will be on the radio, "MLB umps are racist and its better for all playoff teams to have white pitching staffs."

These kinds of studies hurt science in general. It makes good scientists' claims harder to gain footing.

How would you do it?

How would I do which study? The NBA ref one? It will be rigorous. First, I'd start by only including the games that 3 refs of the same race ref. This would be an easy study to gather data, although it may not give a large enough sample size. Then, again this will be rigorous, you'd either have to get the NBA to allow you to see their referee evaluations, or you'd have to watch games to gather the data yourself. Then, I would think you would have a strong data set of truly what race ref called a foul on a player of a certain race. There would be no generalizations and no interpolations.

The IAT study is a little different. I think you'd have to do the study multiple times using different words, different faces, and introducing the relationship different ways. Good + european = right, Good + African = right, Good + african = left, etc. This one is tough because of the practice component and the pattern component. The better bet may be to make 6 or 7 different tests and increase your sample size numerous times.

How would I do which study?

By what method would you classify players and umpires by race?

Re: #80:

Wasn't there a supreme court case a while ago that said the KKK was allowed to do just that, hold a public access TV show?

Can one of the lawyer types around here confirm that?

By what method would you classify players and umpires by race?

I haven't read the MLB study, so I don't know how they did it. I'd probably either try and get the data from MLB or get it from an outside source. Last resort would I go through and label them myself, but if I were forced to do that (and sometimes you are), then you'd have to use a group and not just one be all end all.

Someone brought up a great point earlier about Latin Americans and African Americans. Both have dark skin and could be considered "black" or "African-American" by appearance, so you'd have to make the determination whether to go by skin color or background. This would be a huge influence on the study, because if you really want to go racial bias by appearance, I'd think you'd group Latin-American players with African-American players. Umps aren't going to have a card with the birthplaces of the players in the game to start making their biases on. I know MLB does not consider Latin-Americans and African-Americans in the same group, so this is something that the researcher has to determine at the beginning and state. If the study were done blind, then you could probably do both, considering the two groups together and separate.

Wasn't there a supreme court case a while ago that said the KKK was allowed to do just that, hold a public access TV show?


I don't know of such a case but that ruling is the undoubted outcome of such a challenge.

It should be--free speech means anyone and anything--but I have my doubts.

As mgl points out in #90, the standard error of the 1729 black/black number is very small, about 1%. But if you subtract FIVE strikes from that black/black cell, the discrimination by umpires disappears completely!

So I'm wondering how to reconcile the authors' findings of significance with the fact that the difference is only five pitches, less than half a standard deviation from expected.

See my blog post for details. Any help appreciated, I really don't know how they managed to find significance.

Jimmy - I think I understand your objections now. The original NBA study was done in the manner you suggested, IIRC - that is, they separated crews into 3 black, 2 black 1 white, 2 white 1 black, and 3 white. The findings were the 3 black and 3 white crews tended to exhibit same-race bias, while the mixed crews did not, or did to a much reduced degree. And this was just using the data they collected from boxscores. There's a link on the last page that discusses the NBA releasing their superior data, with specific fouls linked to each ref, and Wolfers is quoted as saying this data will confirm the hypothesis. Again, I'd have to reread the study to be certain, but I'm pretty sure this was the case. The IAT is another story, but I'm not drawing any conclusions from that.

Phil - if that was the major tenet of the study, I would possibly agree with you. But the main point has to be the white umps, since there are so many more of them, and as mgl said, that number is pretty siginficant.

In many of the baseball games I've watched in my life, the game seems to come down to 1 or 2 very close pitches, and this can happen at any time in the game (early or late, I mean). I guess what I'm saying is is that I'm not convinced that a 1 or 2 pitch difference is insignificant.

Further, much of the criticisms of this paper and the sociologists' paper on the NBA seem a bit disingenuous or question-begging. Look, I know most of us at this site think, among other things, that we're GMs-in-waiting, psychologists-without-homes, and sociologists-without-frontiers, however, none of the criticisms of these papers are particularly compelling. The sociologists' paper was peer-reviewed, and subsequent to FURTHER critique that substantiated their methodology. To say "I'll be rigorous" is about as compelling as when I claim in my papers that "I'll go deeper than prior analyses."

And this was just using the data they collected from boxscores. There's a link on the last page that discusses the NBA releasing their superior data, with specific fouls linked to each ref, and Wolfers is quoted as saying this data will confirm the hypothesis.

If that turns out to be the case, I'd have no problem with that. I just have a problem with what they did. It just screamed of being a media whore to me (which wouldn't surprise me considering they're Ivy League).

JC Bradbury linked the paper and the related FAQ above, and I would really recommend that people read them and not rely entirely on the summary in the Time article - especially if you want to question the methodology.

I do wonder how the outcome from the study would change (if at all) if the authors introduced age and/or experience as an umpire as a factor.

-- MWE

Look, I know most of us at this site think, among other things, that we're GMs-in-waiting, psychologists-without-homes, and sociologists-without-frontiers, however, none of the criticisms of these papers are particularly compelling. The sociologists' paper was peer-reviewed, and subsequent to FURTHER critique that substantiated their methodology.

You don't think that questioning the data gathering of a study is compelling?

Just because something is peer-reviewed doesn't mean it's good. JC, you and I both know that sometimes the names and institutions on the top of the paper are just as important as the data in the paper to how critical the reviewers are.

You don't think that questioning the data gathering of a study is compelling?

Just because something is peer-reviewed doesn't mean it's good. JC, you and I both know that sometimes the names and institutions on the top of the paper are just as important as the data in the paper to how critical the reviewers are.

Of course, but I don't find your questions about the data-gathering compelling. They themselves (I'm talking about the NBA study) note the limitations of the approach you describe; it's not like they were unaware of it. Sure, it would be preferable if they could pin every call to every ref and every player, but the NBA blocked them from doing that. So, they took the data they had and drew conclusions that that data supported. Am I wrong about this?

Guts/116, I don't think the number of white umps is relevant. Suppose there were 100 white umps, 100 white pitchers, 1 black ump, and 1 black pitcher. The black pitcher met the black ump once, and his one called pitch was a ball.

Does it really matter how many pitches white umps saw from white pitchers? The fact that there was only 1 pitch in the black/black cell pretty much means you can't conclude anything.

JC - you are correct. The NBA may allow them access to the better data, which will refine the study; again, the link is on the other page.

Sure, it would be preferable if they could pin every call to every ref and every player, but the NBA blocked them from doing that. So, they took the data they had and drew conclusions that that data supported. Am I wrong about this?

No. I find it irresponsible that they knew there were limitations in the data they used, leading to limitations in the conclusions they drew, and they still went ahead and published it. They probably could've gotten the same result had they used data from ref crews of all the same race, rather than saying that 2/3 of a race = 100% of that race. With this data, they would've had a solid, non-assumption based study, and then they could've pressured the NBA to give them the strong data.

Because they went ahead knowing the limitations, I think they were more concerned with causing a media shitstorm than actually looking at the data or trying to determine the deeper meaning (or cause) in the data. Were they doing the science for knowledge, or for personal glory? I vote personal glory.

See my blog post for details. Any help appreciated, I really don't know how they managed to find significance.

Because the null hypothesis (that there is absolutely no difference) is always quasi-false, in whatever field of human behavior one chooses to study. With a large enough sample size, even the smallest differences can be found to be "statistically significant." The real challenge is to determine whether the findings have any practical meaningfulness.

The more I look at this study, the less impressed I am by the magnitude of the findings. If you look at table 2 in the study, you'll see how tiny the differences in called strike % really are. For example, a black pitcher gets the call 30.62% of the time, with these racial "disparities": Wh Ump: 30.61, Hisp ump: 30.77, Bl Ump: 30.76. So, if a black pitcher was judged by a same-race (black) ump 91% of the time (as white pitchers now are), he would gain one extra called strike for every 784 called pitches. A starter has about 50 called pitches per start, so a black starter might get 2 more strike calls in a season. Let's be generous and say that results in 1 fewer walk (or 1 more K) each year (it's probably less than that), in which case he would give up 1 additional run every 3 years or so. The same analysis for Hispanic pitchers yields an advantage about twice as great, maybe 5 extra strike calls for a full-time starter. (And of course, we don't know if that difference reflects positive bias by the 3 (!) Hispanic umps for Hispanic pitchers, or discrimination by white and Black umps).

The authors' own estimate of "same-race" advantage, after a lot of complex regression, is just 0.34%, or about 5 calls per season. So a white pitcher gains perhaps 1 win every 15 years. And that's if you buy their regression results. Using the unadjusted data, a white pitcher who faced only black and Hisp umpires would see a reduction in called strikes of 0.15%, which means he would lose about 2 calls per season or maybe 1 win every 30 years.

Also, although the authors control for a boatload of factors in their regressions, it doesn't appear that they control for park effects. That probably doesn't matter in the aggregate, but it could have a big impact on the Questec/non-questec part of their analysis. Once you start looking at same-race matchups for minority pitchers (for which the sample is small to begin with), separately by Questec/non-questec, you almost certainly could see a real park effect.

The bottom line using this data (Table 2) seems to be that the racial bias — if it exists at all — is limited to Asian and Hisp pitchers. White and Black pitchers get the same called strike rate by umpires of all races. Asian pitchers get lower rate from both Black and Hisp umps (or, less plausibly, a positive bias from white umps). And Hispanic pitchers may get a slight positive bias from Hisp umps, and slight negative bias from Hisp umps.

(I think the only way to really get a good read on this is the questec data, that would allow us to see how umps call pitches in comparison to actual pitch location. That would allow us to see, for example, if Black pitchers really should be getting so many fewer called strikes than white pitchers.)

They probably could've gotten the same result had they used data from ref crews of all the same race, rather than saying that 2/3 of a race = 100% of that race.

Jimmy, from what I understand they treated 3/0 separately from 2/1. They didn't group them together.

J. Cross - that's what I understand as well. See #115.

Phil - the white umps matter because the bias is most easily shown there. The data leads us to conclude they have a significant bias; the question now is can we see the same result in the other, less significant data we have? The answer seems to be yes. The effect is small, but noticable.

Phil - the white umps matter because the bias is most easily shown there.

How can you measure bias by white umps, except by comparing them to the 3 Hispanic and 5 black umpires? And even then, white umps call strikes on a black pitcher just 0.15% less than do black umps. That's about one call per 80 innings, so about one extra BB per season for a full-time starter. And that's ignoring the legitimate sample issue Phil is raising. (The difference is even smaller if you do the same analysis for white pitchers.)

Phil - the white umps matter because the bias is most easily shown there. The data leads us to conclude they have a significant bias ...

Actually, the data can *at most* lead us to conclude that there is a bias SOMEWHERE. It could be white on black, it could be black on white, it could be black on black, or white on white. All you can say is that "same race appears to give different called strike rates than different race," but you can't tell if it's white or black bias, or even which pitchers the bias favors or shortchanges.

For significance levels, it's the SMALLEST cell in the matrix that largely determines the probability of observing the bias. The variance for the sum or difference between two means (such as means of called strike rates) is the sum of the variances of the two samples. The black/black cell has such a large variance compared to the rest that it almost doesn't matter how big the white/white sample is.

Good stuff Phil...both here and on your blog.

I may well be understanding this incorrectly but after reading the article it appears that the statistically significant finding they had was the difference between "matched" umpire-pitcher tandems and "unmatched" umpire-pitcher tandems and that while the individial race-to-race matchups went in the expected direction they weren't statistically significant.

This isn't quite as strong a result as if each of the combinations was signficant (if I'm understanding this correctly) since the matched data is mostly white pitcher-white ump and the unmatched data is mostly latin pitcher-white ump and could (just?) be showing that white umps call fewer strikes for latin pitchers than white pitchers. Since Latin and white pitchers come from a difference baseball cultures and might not be distributed evenly throughout age groups it's probably not a safe assumption that white and Latin pitchers are throwing the same % of strikes.

I like that they controlled for pitch count among other things but wish they had more years of data and, even better, wish MLB would give them access to the questec data.

Jimmy P. et al., regarding the IAT and ordering: it's a legitimate concern, but the experimenters did account for the ordering effect: half the time "good" is first associated with "african-american". I think they accounted for handedness in the same way. If you take the test multiple times, you'll see this. From their web page https://implicit.harvard.edu/implicit/demo/background/faqs.html :

"Could the result be a function of the order in which I did the two parts? I had to group one category together with pleasant words first. I then found it difficult when I later had to group the other category with pleasant words.

Answer: The order in which tests are administered does make a difference to the overall result in some tests. However, the difference is small and recent changes to the test have sharply reduced the influence of order. Because of this order effect, the orders used for IATs presented on this website are assigned at random. For any data we present, we are careful to be sure that half the test-takers got the A then B order and the other half got the B then A order. With the revised task design, the order has only a minimal influence on task performance. If you want to check whether the order made a difference for you, you can take the test again and complete it if you get assigned to the reverse order. If you do take the test twice in different orders and get different outcomes, the best estimate of your result is intermediate between the two. For more information about the order effect, see this paper (Nosek, Greenwald, & Banaji, in press)."

Is the umpire/race effect strong enough that any betting system has ever mentioned it? A story by Dave Sessions in today's Star-Telegram says that "when both starting pitchers or neither starter matches the umpire's race, the home team wins 53.8% of the time. When the home team's pitcher's race [alone] matches the umpire's, the home team wins 55.6% of the time." You would think a factor that large would get considered by bettors, who look at every tiny thing ...

134/Bob: The study shows that white pitchers are better than hispanic pitchers, who in turn are better than black pitchers ("better" means "more called strikes").

There are more white umpires than black, so the team with the pitcher whose race matches the umpire's race probably has a white pitcher. And that's an above-average pitcher, so the winning percentage is higher.

As far as I recall, the study never actually mentions that the white pitchers are better than the black pitchers, so it's easy for reporters to get confused by what's going on here.

I have never heard this mentioned before in betting circles (which does not mean that SOMEONE has not been using it), but I think that most of the differences they find in the win rates are pure random fluctuation.