Oh, and on a tangental point, because the threshold for good #2ing is so surprisingly low, and because injury is heavily affected into it, there will be lots of pitchers who are projected to be decent #2s but no rotation in baseball (well, virtually no) would call them in #2 in their rotation (providing they had any sort of staff). For example, should the Mets have given up Heilman for Jaret Wright, even though on last year's ERA+ (and give him a point or two for leaving the AL) he's a below-average #2? So you can still be a #2 and surplus to requirements.

Again, I think this is great stuff, but I'm not sure it helps us get any better at valuing contributions from pitchers, because the overriding factor that decides rotation spot and production is injuries.

This is just as wrong as the THT times article for just the same reasons. The crucial, argument destroying, point, has been made over and over again and people refuse to see it. Frustrating.

Anyway while this article does demonstrate that replacement level starting pitching is really bad it unequivocably does not demonstrate that #4 or #5 starters are worse than people think. In fact given the reaction to both of these articles I would now say that #4 and #5 starters are much better than the people in these threads think. Both of these articles are poorly thought out and completely misleading. However given the way bad ideas have a way of spreading around here I have already seen people use these kind of arguments to argue that Jason Marquis is actually not a bad fifth starter despite the fact that he is the worse starting pitcher currently in a five man rotation.

This is just as wrong as the THT times article for just the same reasons.

What is the shared flaw? The THT article ranks pitchers by actual ERA performance, which is misleading at both the high end and low end, as MGL argued. But isn't the approach here of grouping pitchers by their 'slot' quite different? It's not using actual performance to rank pitchers. In fact, it seems to me this approach probably overstates the ability of true #5 starters, in that teams will sometimes put pitchers in the 5th slot who are not in fact perceived to be the 5th best (such as a promising rookie). Why do you see both articles as having the same problem?

This, and the THT article which is not as good as this, demonstrate two things: (1) replacement level pitching (i.e. 6th, 7th, 8th starters) is even worse than we thought and (2) #1, #2, #3, and #4 starters ARE worse than we thought because they keep getting hurt all the time.

Yes, in terms of "true talent" or ability, Jeff Suppan is probably a low #2, high #3. Heck, maybe he's a low #3. But Jeff Suppan has made 30+ starts and thrown 188+ IP for the last 8 years. In the past at least, he has been way, way, way more durable (and consistent) than almost all the starters in baseball. Past durability may or may not be predictive of future durability. If it's not, then sure teams should make FA starter decisions based heavily on "true talent"; but if it is, then Jeff Suppan is pretty clearly a #2.

Which gets back to Darren's idea of looking at VORP or a similar statistic which combines quality and quantity of performance. Unfortunately VORP's a pain for us non-subscribers to work with so I'll use WARP3 for my cherry-picked examples (alas WARP 3 includes hitting for NL pitchers). I don't know who folks think of as a "true #2" but I'll agree that Suppan has way less talent than AJ Burnett. Over the last 6 years, Burnett has 23.1 WARP3; Suppan has 26.7. Over the last 6 years, Penny has 24.3. Kerry Wood, quite healthy for 2001-2003, has 28.3. Over the last 5 years, Beckett has 22.2, Prior 22.3 and Suppan 21.3.

Based on past performance, Jeff Suppan is gonna give you 4-5 WARP every year. Lots of "true #2s" (and even #1 talents as many of those above guys are) are either gonna give you 6.5 or 2.5. Given that in those injured seasons you're likely to replace their innings with a guy with an ERA of 6+, that "slot" is going to seriously underperform Suppan's "slot" in those years and often come out about equal over a number of years.

This isn't to suggest that Suppan's numbers are outstanding even for a "#2". Over the past 6 years, even with injury problems, Pettitte has 35.5 (though that might be #1 territory).

The key question about Suppan isn't his true ability level but whether his past durability is predictive of future durability. There are obviously other concerns such as his age and whether that along with relatively low K rates is predictive of substantial decline in his ability. But if the probability that Suppan, over the next 4 years, will give you 125 starts and 750 IP is way higher than the chances that Lilly or Meche or Schmidt or Beckett or Burnett, etc. will, then he's a pretty valuable property and most likely a "#2" starter.

And that's gonna be true no matter how you look at "real slots". Maybe the reality is you can expect 25 starts from your #1, 25 from #2, 25 from #3, 25 from #4, 25 from #5 and 37 from replacemets. If you can count on a "#3 ability" pitcher taking 5-7 starts away from the replacements every year, that pitcher is most likely gonna be as valuable as the typical "#2 ability" pitcher in his 25 starts plus 7 replacement starts.

I think this is a very important point, with the big caveat that we don't really know how predictive past durability is. GMs seem to give it some weight (K. Wood), but not as much as you might expect.

BTW, if your point #1 is correct, this makes the Suppans (avg. talent, high durability) even more valuable than the WARP numbers suggest. Let's say replacement level is really more like 6.5 than 6.0 R/G (30% above average rather than 20%). In that case, all these pitchers gain value in direct proportion to innings pitched -- roughly 1 win per 160 IP -- which will most boost the value of those with the most durability.

Nicely put, Walt. When you consider Suppan retrospectively, he has performed at the #2 starter level (or thereabouts) for the past several seasons. However, people care about the future when they sign players, not the past. Compared with a guy like Beckett or Burnett, all Suppan has going for him is durability. A nice thing to have, don't get me wrong.

Has anyone ever studied how likely durable pitchers are to stay durable in the future, especially compared with "flaky" starters? If not I might try to some night this week.

Awful flattering to see this thing keeps gurgling back up on occassion . .

Taking a general look back on this thread, the question is when is this study useful and when is it useless. For examining individual pitchers, it's useless. A lousy pitcher who is a good fifth starter is still a lousy pitcher.

Another danger spot comes when how replacement starters can warp findings. For example, Brett Myers missed two starts, and his replacements did dreadful, lowering the Phillies' ace hole. Meanwhile, Pettitte misses a start and his replacement throws a shutout. While that does tell us something about the quality of the team's back up starters, there is a sample size fluke going on. This can screw up the numbers for a given team. While one team's sample might get screwed up, I think overall there's something useful here. In the four leagues looked at, the league-wide ERA+ for the different slots is largely the same everytime. The 2005 NL is a bit higher, and things are a little different in the ace slot (The Royals drag down the AL), but if you have a pitcher who can give you 33-34 starts, 210 innings, and an ERA+ at 122 or so, that's perfectly typical production from the ace slot.

Compared to the Sackmann study, which I find really cool also, I think the main advantage of this one is that this does more to factor in durability. Your worst 32 starts ain't all in one slot but generally spread out in the rotation doing fill-ins in several slots, hence making a man like Suppan a little more important than one might guess. Sackmann gave his study's results in ERA. Here's this study in terms of ERA (slots 1 through 5):

2006 AL: 3.86, 4.43, 4.64, 5.14, 5.79
2006 NL: 3.66, 4.20, 4.70, 5.21, 5.82
2006 MLB: 3.75, 4.31, 4.67, 5.18, 5.81

Chris:
Given your methodology, about what percentage of the time would you say the pitcher in the 5th slot is the pitcher perceived (by the team) to be the weakest of the 5 starters in the rotation at that moment? Do you think that's the case for 75% of the 5th slot starts? 90%? (Apologies if you've already answered this above.)

Given your methodology, about what percentage of the time would you say the pitcher in the 5th slot is the pitcher perceived (by the team) to be the weakest of the 5 starters in the rotation at that moment?

Extremely difficult to say. First, I only follow two teams (Cubs, Sox) close enough to say how their slots are perceived by their teams. Second, the difference betwen the #4 and the #5 slot often isn't that great. Looking at it, the difference between the two slots was less than 0.25 ERA for 8 teams, & between 0.49-0.25 for six more. In many of those cases, it's perfectly legitimate to not have a clear idea what the fifth slot is. (If you're curious, among the other sixteen teams, exactly half had the difference greater than 1.00 ERA and the remaining eight were between 0.99 and 0.50). Also, there are several problems in trying to determine perception vs. reality. Let me use the Sox as an example.

Last year, their #5 starter according to this was Mark Buehrle. Now, he wasn't considered the #5 guy largely because he was so good prior to this year, and frankly I'd agree that prior performance is a valid thing to take into account. That's especially the case because there was so little difference betwen him and #4 starter Vazquez (0.11 in ERA). Heck, he was only 0.61 behind the ace.

Finally, for much of the year Buehrle wasn't pitching like the #5 guy. He was doing pretty good until an epic mid-summer meltdown. People were definately aware how bad he was pitching in that stretch, and knew he was the worst starter at that period, but that's a little different from saying he's the fifth starter for the full season.

In fact, the fluxuating performance out of the fifth slot is common as that's almost always of revolving door of sucktitude. Making it even more confusing, the #4 slot is also often a revolving door of sucktitude. For most teams, what qualifies as the fifth best starter changes from month-to-month. And it's almost impossible to know which slot did the worst over the full season because there are so many changes in the course of the year who really knows or remembers all the pitchers in a given slot.

For example, the Red Sox fifth hole consisted of Wells, DiNardo, Pauley, Lester, Snyder, Hansock, and Gabbard last year. Meanwhile, their fourth slot had Clement, Snyder, Johnson, Wells, and Tavarez. Can't tell the players even with a scorecard. The fourth slot had a 5.57 ERA and the fifth slot a 5.70. I'm sure Red Sox nation knew their backend stunk, but I don't blame in the least for not being able to say which was fourth and which was fifth. Heck, after Wakefield went down their third slot consisted fo Gobbard, Johnson, and Snyder -- all of whom also pitched in the bottom two slots. In situations like this divvying up slots is more art than science. And it really points out a central artificiality to my whole approach.

Looking it up, I'd say teams know who their fifth slots are when it's reasonable to know that.

FWIW, the teams with the biggest gaps between their fourth and fifth starters were the Yanks, Jays, Mariners, Mets, Marlins, Brewers, Cubs, and Giants.

The Yanks had no more than 11 starts (Chacon) by anyone in their fifth slot. Cory Lidle ended the year in that slot and was put in the bullpen for the playoffs.

Toronto's fifth slot had 12 starts from Josh Towers, and 14 from Marcum, along with various replacements. Their fourth and fifth slots were also in disarray, but wow that Towers sucked.

Seattle had a very stable rotation, but Joel Pineiro (remember when he was good?) was horrible. After the squad traded Moyer, though, there wasn't much difference between the fourth and fifth slots.

The Mets fifth slot I cover in the article. No one with more than 8 starts. Their fourth slot had five pitchers in it, most of whom really sucked, but El Duque was alright. When he wasn't in, there was reason not to know what the fifth slot really was.

The Marlins are strange because their fifth slot changed over the course of the season. Early on it was Brian Moehler, but he got replaced by Sanchez, who was fantastic. That ended up being the fourth slot as the Mitre-Nolaco-Mohler combo did far worse. Nolasco was OK, but not nearly as good as Sanchez, while Moehler-Mitre combined for 40 ER in 55 IP. The fifth slot was wherever Brian Moehler was, and it looks like the team knew that, given how they jerked him out of teh rotation.

Milwaukee's fifth slot looks pretty obvious. Check that. Again, their fifth slot changed in the season. Early on the worst performance was in what was the season's fith slot. Ohka had 18 starts and was clearly worse than Sheets, Capuano, Davis, or Bush. He went down and was replaced by four men. The first three were truly wretched (a combined 39 ER in 23.3 IP - GACK! - By Hendrickson, de la Rosa, and the undead Rich Helling. When they finally found someone semicompetant in Villanueva, he outpitched the fourth slot for the last month.

The Cubs I covered in the article. Their fifth slot had an ERA of 7.71, fourth slot of 6.31, and third slot of 5.39. Speaking as a fan, though, it was almost impossible to tell the difference between the last two slots, though. One was a slot dedicated to waiting for Mark Prior to become unhealthy enough so they could pretend he was healthy, and the other was dedicated to the same thing for Kerry Wood. The main difference is that the fourth slot had Carlos Marmol allow "only" 45 ER in 67.7 IP over 13 starts. That's enough to set it apart from Rusch/Prior/Ryu/cyanide, but from the fan's perspective you really can't tell the difference when it's a matter of being 2 runs per game below replacement level or merely 1 run. And both slots were a revolving door with at least a half-dozen pitchers each. Making it even more confusing: the third slot was even worse than both of them after Maddux got traded.

Finally, the Giants. Looking at it, I certainly hoped the Bay Area faithful notcied Wright was their worst starter. I can only assume they noticed his replacements (mostly Hennessey, but also Sanchez and Fassero) were even worse. It really was the fifth slot all year long.

Getting back to your question, I assume teams know who their worst starter of the moment is, but that changes considerably as the year goes on. Heck, look at the above examples -- those are the teams with the greatest difference between 4th and 5th slots, and even there it usually changed.

Teams usually know which pitchers need to be replaced, and really that's the only important thing. If you have 3 pitchers who need to be replaced who really cares if you think of the fourth as the fifth and vice versa? The outstanding exceptions to this is the 2005 Royals & Reds, who left Lima & Milton in all year while guys pitching better were jerked out of the rotation. Even then there's mitigating circumstances (both had established track records, the entire Royals staff stunk, and Milton had just signed a big contract).

Thanks. It sounds like a team's worst two pitchers, at an given time, would pretty consistently be assigned to slots 4 and 5. Is that a fair generalization?

And let me pose a related question about the methodology: if the #2 starter goes down for the season on June 1 with an injury, will the new starter go in his "slot" so long as he follows the #1 guy in the rotation, even if he clearly is the new #5 in terms of ability? Or do you bump the others up a slot and assign the new guy to slot 5? And if it's the former -- based strictly on rotation sequence -- would you say teams are pretty quick to reorganize the rotation so that rotation order follows ability/performance, or are weak pitchers often allowed to remain in the 2 or 3 'slot' for extended periods?

Thanks. It sounds like a team's worst two pitchers, at an given time, would pretty consistently be assigned to slots 4 and 5. Is that a fair generalization?

Yup. For every team that has a stable fifth starter, there are multiple teams with no fourth starter, and some teams without a real third starter.

And let me pose a related question about the methodology: if the #2 starter goes down for the season on June 1 with an injury, will the new starter go in his "slot" so long as he follows the #1 guy in the rotation, even if he clearly is the new #5 in terms of ability?

Yup. I don't assign a slot ranking until the end. First I just figure who replaced who as best as I can, then add up their numbers, then label them "first slot," "fourth slot," or whatever.

Take the Twins, for example. Liriano ended up in their third slot despite pitching with a sub-2.00 ERA as a starter. Initially, they had Carlos Silva, who allowed 45 ER in 46 IP in 8 games. Then came Liriano for 16 starts. Then they ended the year with Mat Garza pitching poorly. Making it that much more confusing, Silva spent almost no time in the bullpen. He came back very quickly and replaced Scott Baker in the rotation. Mind you, Baker pitched better than Silva, but Silva had teh better track record, and it fact he did pitch better the rest of the way than Baker had earlier. But don't cry too much for Baker, he got a shot back in the rotation later than year, allowing 16 ER in 14.7 IP in the fifth slot before getting bounced. He then came back and finished the year when Radke went down, pitching reasonably well.

Like I said, it doesn't really make a difference if you define one guy as your fourth starter and one as your third, just as long as you know who is pitching bad enough to earn a demotion and who isn't.