Some of the common critiques of the Wins Above Replacement framework include: 1) Why do FanGraphs and Baseball-Reference.com have such different numbers, 2) How can we trust it when the numbers change, and 3) How can we trust it when I can’t calculate it?
For the first question, our announcement today of a consistent replacement level between FanGraphs and Baseball-Reference.com has done a considerable amount to bring our two methodologies into alignment at least on the question of how big of a basket of WAR to hand out to players each year. Previously, FanGraphs allotted nearly 300 additional WAR due to a much lower replacement level. Our meeting in the middle has erased this difference to zero.
For the next two questions, I would point to a very widely quoted and very widely used statistic from economics, Gross Domestic Product (GDP). Here is the Wikipedia article on Gross Domestic Product. I’m going to argue that WAR is essentially GDP for baseball.
8 - The people who compute and create GDP calculations are economic experts who are building on years and years of economic study and research. The people who compute and create WAR or WAR-like frameworks are building on and expanding the years of sabermetric research by experts now employed directly by teams (Sean Smith, Tom Tango, Keith Woolner, Bill James) as advisors or experts in the area of statistics and evaluation (Nate Silver, Pete Palmer).
9 - The people who use and rely on GDP, news media, politicians, business owners don’t have a prayer of computing it, but rely on subject experts to provide well-reasoned and carefully calculated estimates of economic value. The people using WAR (GM’s, news media, agents) to estimate player value don’t have a prayer of calculating it, but rely on subject experts either publicly or privately to provide well-reasoned and carefully calculated estimates of player value.
I can certainly understand unease with using one-number estimates like WAR, but I would point out that it comes from a long line of research, thought, and process that is common throughout the social sciences.