Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Primate Studies > Discussion
Primate Studies
— Where BTF's Members Investigate the Grand Old Game

Tuesday, January 14, 2003

The Particle in the Box

The eternal debate on where to pin that infernal replacement level continues.

Nelson Lu does the baseball internet community a wonderful service.  He compiles RC/25 and posts it on rec.sport.baseball.  I understand it is done elsewhere around the ‘net, but Nelson does it in a wonderful discussion forum.  I don’t know if he makes his newer lists available on the web anywhere, but they are very interesting.  For 1995-1997, see www.baseball1.com.

Nelson has a slot for “league average” and a slot for “replacement level” in the lists.  We all understand average, but “replacement level” is a more nebulous, ethereal mark.  Nelson defines replacement level, for his lists as 30 pts of OBP and 30 pts of SLG for RC/25 or 50/50 in RC/27.  The reason is that in-season reports are RC/25 and only use regular players, while the end-of-season reports are RC/27 and include all players.

Like any good researcher, I looked up other “Replacement Level” (RL) definitions.  I really like VORP.  I think it is the best up-to-date player evaluation system out there.  So I went and checked www.stathead.com.  It was there that I read that Keith had done research to actually see what replacement level is.  Well, that’s what the purpose of this article is, but we are going to do it slightly differently.  Keith wrote a long article in Baseball Prospectus 2002 demonstrating his replacement level at 80% (85% for catchers and 75% for 1B/DH.  He used EqA for everyone who wasn’t a starter.  It’s a great article - gee, what a surprise.  Interestingly, Clay Davenport doesn’t use that for replacement level about two mouse-clicks away from Keith’s VORP.  At the BP Stat report site, Clay uses 0.8835*EqA at a given position.

League average is 330 OBP and 410 SLG.  So Lu RL is 280/360.  Woolner RL is 264/328.  Davenport RL is 290/360.  These really don’t miss each other by much.  I suspect only Keith’s comes from in depth research, and it is much lower than the others.  Evidently, the pickings get slim.

Unfortunately, I started my research before reading that Keith had done his work.  I’ve long been annoyed with the generalized value for replacement level, when a concrete answer should be available.  As it is always a “guess”, Lu and Davenport are really “close enough for government work.”  Probably.  Since I did the research, I’ll present it - it will serve as either confirmation or disagreement - but it will be done as a researched value.

Replacement Level is generally defined as “freely available talent”, where “free” just means “very little money put forth.”  At least that is how I learned it.  That is important to me because Keith’s article doesn’t use that.  As you can follow on Clutch Hits there is considerable opinion regarding replacement level, and that discussion prompted me to finish this and get the data available.

Dan Szymborski and I gathered a long list of players that actually were freely available - or appeared to be based on free agency signings and other small things.  I gathered the EqA for each player over the last three seasons (1999-2001).  Remember, I started this project in July?  I took the three-year average for each player.  Sorted into positions, I averaged the EqAs for each position.

Here are the results:

Position (sample)    Avg   Median
Catcher (14)      0.236   0.239
First Base (13)    0.253   0.256
Second Base (10)    0.246   0.242
Third Base (12)    0.241   0.241
Shortstop (11)      0.224   0.219
LF/RF (34)        0.250   0.248
Centerfield (14)    0.242   0.242

Yes, that only stocks one league, but I struggled to get all the EqAs.  Dan actually supplied with a list of 20-30 at each position, but you go with what you have.

Clay Davenport’s EqA 2002 year-end report lists the EqA by position:

Position     EqA   RL
Catcher       0.246   0.217
First Base     0.287   0.254
Second Base   0.259   0.229
Third Base     0.264   0.233
Shortstop     0.256   0.226
LF/RF       0.283   0.250
Centerfield   0.270   0.239

Clay’s system does a nice job for some positions and not so good for others.  He is using a constant multiplier for ease and it does pretty well.

Comparing Clay’s RL to my averages, first base, shortstop and the entire outfield are good matches.  Second base has a few good players that no one seems to want, but 0.229 is definitely too low.  The lowest 2B in my group is at 0.236.  Third base is definitely too low - Ryan Minor epitomizes “replacement player” and he has posted better EqAs than that.

Catcher is amusing - the group of catchers in my study actually performed nearly as well as the actual 2002 major league group.  I did have a few below 0.217 (Clay’s RL), but the median for my group was 0.239.  Again, these guys were selected because they had recently been had for free.

In summation, offensively, replacement level appears to be slightly higher than Clay’s 0.8835 is set, but he has a very good replacement level set for a generic adjustment.  Digging up the additional data for the rest of my data set could tweak the values either way - the median numbers were very close to the averages.

Replacement level for all positions is about 90-91%, with catchers being much higher (95-97%).  I suspect this is due to the defensive premium placed on catchers - sacrificing bat for arm.

Chris Dial Posted: January 14, 2003 at 06:00 AM | 16 comment(s) Login to Bookmark
  Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. MattB Posted: January 14, 2003 at 02:22 AM (#608349)
Am I correct in assuming that when you say "Replacement level" is about 90%, you mean 90% of average?
   2. MattB Posted: January 14, 2003 at 02:22 AM (#608351)
If so, this strikes me as very odd.

Assuming a group of 30-60 people at each position (starters and backups) who are considered above the freely available level, I would expect the range to be greater than 10% in either direction.

If the not-free group ranges from 85% to 115%, say, it strikes me as odd that the value of "free" would be so high.

Perhaps by only looking at a dozen or so players for each position, you are really tapping into the inherent inefficiencies in the market. Perhaps the 10 second basemen who are "freely available" (or even half of them) are those who are blocked by the best starter/backup combos in the league. Essentially, there will always be a bunch of "freely available" talent that is above "replacement level."

This is not necessarily a bizarre conclusion. Six year free agents and the like are only available in the off-season, where they flood the market. I can replace my disaster of a second baseman with "freely available talent" in December, but in June I'm stuck with whoever's holding down the fort in AAA, who is likely a worse option than will be available in the off-season.

For example, I can probably sign Marlon Anderson for near-league-minimum today. If I wait until July, I probably don't have access to anyone nearly as talented. On the other hand, I may think I have a guy now who looks to be better than Anderson. By July though, he could be sucking, injured, or crushed by a meteor. By then, I have better knowledge that my player in not better than Anderson, but I've missed my opportunity to sign him.

Essentially, I think the quality of a freely available player will vary over the course of a season (better in the off-season, worse during the early season, maybe improving again later on as teams try to sneak quality players through waivers.) So it is difficult to determine "freely available" just by judging who was actually signed.

Assume two players. Player A is Marlon Anderson (a 95% player, say). He is signed tomorrow by some team as is included as "freely available" in your chart. Player B is a AAAA scrub who is not anywhere near as good as Marlon Anderson (say, an 85% player), but he is the best second baseman available in July after Anderson and his ilk have all been inked. My second baseman, who is currently being paid $800K is unexpectedly sucking in July (playing at 86%), playing much worse than Anderson, who signed for $300K. But the best available replacement is Player B. I don't think Player B will play any better than my sucking player, or maybe he'll be just as good (or only a tad better), so I don't sign him (and pay $300K more on top of the $800K I still have to pay my guy). Player B is the best freely available talent, but he doesn't make the list of freely available talent because he is never signed.

If I am right and freely available talent is "better" in some parts of the year than at others, then there is a selection bias in only using the freely available players who are actually signed (in the high parts of the cycle), but leaving out those who are passed up during the talent troughs.
   3. Chris Dial Posted: January 14, 2003 at 02:22 AM (#608352)
Yes, Matt, 90% of average. I think it really suggests that MLB teams do not realize they could get the same production for a lot less money.

Actually, that is an error in my methodology. I shouldn't use "MLB average", but rather the average of the MLB starters. The MLB average includes some of the replacement players - but not all. The OF for example includes Jose Canseco and Ozzie Timmons. Sure, Canseco drives up the replacement value, but isn't he freely available? Wasn't he all of 2002? He sure as heck could have helped the Mets. And remember, Jeromy Burnitz played so poorly with about a million PAs, he is going to drag the average down.

But yes, 90% of average. It does seem high, but Clay already uses 88.35% of average.

And I think most of these players are freely available through most of the season. It wouldn't have taken much of anything to get Matt Franco from the Braves.
   4. MattB Posted: January 14, 2003 at 02:22 AM (#608354)
Aside from what the right question of "average" should be, the "freely available level just looks too high.

Take, for example, second baseman. "Freely available" is calculated as .246 EqA (.242 median). Taking the lower number, only 34 major league second basemen surpassed that level (based on BP's calculations). That number includes Bry Nelson (38 PA) and Brandon Phillips (31 PA). None of the other 33 listed player broke .242 (4 hit in on the nose). If you set "freely available" at .230 EqA, you get the number up to 43.

I am prepared to be convinced otherwise, but I find it initially incredible that only 34 major leaguers are above "replacement level" for a second baseman. I can accept that there is some inefficiencies at the margins, but not that over 3000 Plate Appearances went to below Replacement second basemen. The market is not THAT out of whack.
   5. David Nieporent (now, with children) Posted: January 14, 2003 at 02:22 AM (#608359)
MattB:
I can accept that there is some inefficiencies at the margins, but not that over 3000 Plate Appearances went to below Replacement second basemen. The market is not THAT out of whack.

Don't forget, Chris is only looking at offense. Presumably, hopefully, most of the really bad hitters are actually above replacement level once you include their gloves.

.
Also, it's not all inefficiencies; some teams play poor prospects, knowing that they're poor, because they're prospects to whom teams want to give major league experience. Just because I can find a 30 year old AAA journeyman who's better right now than my young 2bman, doesn't mean I'll play the former unless I'm Syd Thrift.
   6. DanG Posted: January 14, 2003 at 02:22 AM (#608361)
Replacement Level is generally defined as "freely available talent", where "free" just means "very little money put forth."


This is not the definition of replacement level that I am most familiar with. It is more along the lines of "What level does a player have to be at to make him be replaced (lose his job)?"

Defining RL as you have leads to a higher replacement level, I think. You have to be showing them you're better than what they have to be worth picking up.

Defining RL as I suggest leads to a lower replacement level, because of inertia and contracts and other things, teams will hang on to a player who really sux for awhile before they give up on him.

I'm saying that for a guy to be picked up from the freely available talent pool, he needs to be seen as better than the guy he's replacing. So where is the right place to put RL? At the "Replaceor" level (as you suggest) or the "Replacee" level? Can another study be done looking at guys who were replaced by freely available talent?


DG
   7. John M. Perkins Posted: January 14, 2003 at 02:22 AM (#608363)
For the big studies, particularly multiple season studies, I understand the use of percentages. But for a single season or any point in time, why use a percentage? Somebody is the 30th best hitting player at a position. To me that is replacement level.

My best argument, in my shallow analysis, is SS. ARod, Tejada and Garciaparra skew the average. Nobody is trying to replace them. If they were closer to the bell curve, the average EQA would drop, but #30 [St. Rey?] would still be #30, and still be the guy you want to replace in Tampa Bay. Since St. Rey defines replacement level, take ARod out of these calculations.

For long term studies, compare the average #30s against the average league EQA at a position and come up with a magic percentage.

But today I want to know who Tampa Bay could get as a NRI that could challenge St. Rey, Einar Diaz, and company.
   8. Michael Humphreys Posted: January 14, 2003 at 02:22 AM (#608364)
Great work, Chris. I think your discovery that freely available players have a surprisingly high value is consistent with Bill James' discovery way back when that teams systematically overestimate the "risk" of promoting a minor leaguer. Inertia and ignorance protect potential "Replace-ees" and keep out potential Replace-ors". If further studies confirm that "freely-available-talent-replacement value" is close to 90% of average value, ratings based upon "average" value might turn out to be not terribly distorted in most cases.

In some sense, your discovery should not be so surprising because, as someone pointed out, "average" value already incorporates a lot of marginal "replacement level" play.

I suspect that the supply of freely available *pitchers* is not so great, and that the difference in value between freely available pitchers and average pitchers is pretty big. Any thoughts on that point?
   9. Chris Dial Posted: January 14, 2003 at 02:22 AM (#608371)
MattB and DanG - it surprised me. I went into the article thinking that replacement level was lower than or close to the 80% that Keith came up with. You heard that correctly. My preconceived notion was that replacement level was too high. I've argued it many times on USENET. Of course, I was always arguing about St. Rey. And the replacement SS do suck. I was also surprised at the average EqAs that Clay had posted. They were lower than I thought.

I looked at Clay's lists and selected the position players that had 300 PAs. There was usually a significant dropoff in playing time after that (30+ PAs) between players. Here are the average of just the top 30-35 - the starters:

C 0.248; 1b 0.288; 2b 0.259; 3b 0.266; SS 0.256 (0.251 w/o); LF 0.284 (no Bonds); CF 0.272; RF 0.285; DH 0.292

So the starters are pretty close to the average. Of course, Clay probably weights his average, so that makes sense.

I assume I looked at the same list Matt did, and I get 40 players with a .241 EQA or higher. And there are only 3 players between 0.230 and 0.241.

I think the explanation is that MLB GMs don't recognize that there is freely available talent that can play as well as expensive players they preently have. I mean Vinny Castilla makes $4.5 mil a season, and there are a dozen FA players that would out-hit him.

David also offers two other reasons - young players getting on-the-job training or strong defensive players that can produce "above average" overall performances, like Pokey Reese.

But from an offensive perspective, the talent that is walking around can hit.

I have a long list of pitchers, but that's another story I'm still working on.
   10. Chris Dial Posted: January 14, 2003 at 02:22 AM (#608372)
Dang, Paul, you posted while I was working on my other response.

Please, don't apologize for your critiques. That is *exactly* what I want to read. The research isn't about me and my ego - it's about searching for an answer - a good answer.

I may have mis-typed - Dan and I didn't look for the "best" freely available talent - we looked at basically any player that was freely available and then I gathered all the data available.

I do think that there will be some fluctuation in the raw RL EQA, based on up years and down years, but won't that happen every season? League BA/OBP/SLG fluctuate, but hover in a close area - well, it did before 1994.

As for the defensive aspect, I don't find that worse hitters are inherently better fielders - that is one of the myths of baseball. Plenty of poor hitters are poor fielders. I'll look at the guys I used and assess their defensive abilities (using the IMO scale).
   11. tangotiger Posted: January 15, 2003 at 02:23 AM (#608375)
The scales being different is a big problem, and does lead people into the wrong conclusion.

   12. Rickey! trades in sheep and threats Posted: January 15, 2003 at 02:23 AM (#608376)
'I've long been annoyed with the generalized value for replacement level, when a concrete answer should be available. As it is always a "guess", Lu and Davenport are really "close enough for government work." '

You know, when the phrase "close enough for government work" was originally coined, it was a compliment (or at least a backhanded complaint) paid by laborers not used to the exacting standards of governmental engineering projects. When a carpenter or metal worker would say "that's close enough for government work" he actually meant "that's so close to perfect that even the Army Corps of Engineers would accept it." That the phrase has devolved to the point of meaning "it's not right, but the government would accept it regardless of its flaws" might be an arguement for the fall of western civilization, if you're being hyperbolic.
   13. Ron Johnson Posted: January 15, 2003 at 02:23 AM (#608377)
Nice work Chris.

Having gathered the data it might be interesting to graph it.

Bill James has long proposed that baseball talent is the extreme
right side of a normal distribution. If he's right (and what you
and Keith have come up with seems to support that notion) then
I suspect that there should be a virtually limitless supply of
players about as good as the 27th best regular. Within method
error.
   14. Chris Dial Posted: January 25, 2003 at 02:25 AM (#608538)
Whew, tough crowd.

Actually, I'm working on another research project. My fault. Your response is probably why most articles on the web don't allow for Q & A. It's tough to do, and silence is regarded as "guilt".

I do apologize for not checking the bottom of this article late Thursday or Friday. I also thank you for being interested enough in my response to speak up (although you could have emailed me ;-) ).

My methodology is fine. My conclusions could be incorrect, though I don't think so. That's part of the point of peer review. As I said, I expected a different result - just because my data showed *me* to be wrong should not keep me from publishing the data.

Did I "botch" the scales? Partially. My direct comparison was to Clay's 88.35% of EqA (which isn't botched). I get higher than that - which is what I based my conclusions on.

Did I "botch" the comparison to Keith's number (80%)? Certainly - but David Smyth corrected it clearly. I used "replacement players", where Keith used "every non-starter". My results would be expected to be lower. Keith uses Erubiel Durazo as replacement level - I do not. I did not make that comparison correctly. I don't think I made an incorrect conclusion based on that.

Now, to answer the question I left hanging:
Using the IMO scale* of 1 (poor) to 5 (excellent), here are the average values at each position. There were a few players who were left out because I did not rate their defensive skill due to my own ignorance. No number is better than a wrong number (I can't emphasize that enough).

C: 3.12 - slightly above average defensively. This is a tenuous rating because of the difficulty in catcher defensive ratings.

1B: 2.85 - slightly below average - several of these guys are DHs, or should be.

2B: 3.14 - again, slightly above average defensively. Of course, one of the 4's is Keith Lockhart...

3B: 2.71 - solidly below average - several of these guys are on the cusp of playing 1B.

SS: 3.11 - slightly above average - good field, no hit. Shocker.

CF: 3.23 - best defensive replacements - some good fielders, just not many bad ones.

LF/RF: 2.83 - slightly below average - they range from poor to very good.

It appears that the hitting position replacements, 1B, LF/RF, 3B, are worse defensively, while the weak hitting positions, C, CF, 2B, SS, are better defensively. And second base is probably the most over-paid position in baseball.




*Granted, you have to take the IMO scale with a grain of salt, but, IMO, I am an excellent judge of defensive abilities. (Sorry about that).
   15. Marc Posted: January 26, 2003 at 02:26 AM (#608545)
The discussion makes it clear why "RL" is so elusive. 1) As Dan G. pointed out, the replacer level and the replacee level are two different levels. And 2) this is only offense. Add in the defense, which quite obviously the decision-makers do even at the corner positions, and your list of RL players (both replacers and replacees) might be entirely different players than the ones this research focused on. Until you can quantify the defense, you don't know if your sample is really RL or not.

All of this aside, isn't it logical that RL (replacee level) is near to 100% of the average, because so many players cluster around that level and you never know which RL player is going to have an abnormal(ly good) year or month or week because of the randomness of such things. So there are always guys who are out of a job who nevertheless have some tendencies (some evidence, some numbers) that seem to put them above RL (tending towards the average).

The only real surprise is that more catchers cluster near 100%. That is exactly the one position (OK, that and pitcher) where I would have thought there would be fewer toward the average.
   16. Chris Dial Posted: January 27, 2003 at 02:26 AM (#608562)
Marc,
you are right if the goal was for extreme precision. Your points are well taken, but I think wrt the moving of the "average" that the replacees become replaceable mostly due to cost.

If a player performs close to the average RL for a couple of seasons(as outlined by me or Clay), and he costs $3 mil, he should be replaced, because you can get *approximate* offensive value for a song. Of course, one does have to weigh the defensive aspect to it. Quantifying the defense, while preferable, is next to impossible with minor leaguers. The key to RL is the cost involved. I understand your point about replacees fluctuating, but I mean to be talking in the general area. Clay has used his RL for years, and my research supported his general number well - when I used specific players. This indicates that the RL while it certainly will fluctuate as you say, the fluctuation will be small.

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Tuque
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Syndicate

Page rendered in 0.4624 seconds
47 querie(s) executed