Another example is Roy Oswalt's 2012 season. 3.27 xFIP, 4.23 FIP, 5.80 ERA. If Cameron is right, teams should be falling all over themselves to sign Oswalt right now. They aren't, though, because his ERA is more indicative of how he performed in 2012 than his FIP or xFIP are. He was throwing meatballs, and they got crushed.

Jeremy Bonderman is another pitcher who seems to fall in the Ricky Nolasco category...throughout his career, he consistently had a significantly higher ERA than FIP.

Quite frankly, I don't know why anyone for rely on FIP anymore

Projection systems are more accurate for predicting future ERA obviously, but that's not always what you want to look for. FIP shows how a pitcher pitched when you look at things not effected by defense. As a by product it also strips out a bunch of luck and a little skill. I personally think it's a much better indicator of in season performance than ERA. Over 6+ seasons, adjusted RA is the best thing obviously. Looking at components seperately is just dancing around the fact. How do you compare a guy with 5 K/9 and 2 BB/9 and 9 K/9 and 4 BB/9? FIP is way better than just saying "well they seem about equal".

They aren't, though, because his ERA is more indicative of how he performed in 2012 than his FIP or xFIP are. He was throwing meatballs, and they got crushed.

Yeah it's pretty likely that he was throwing a bunch of meatballs while also striking out 9 guys per 9. That's usually what happens.

Yeah it's pretty likely that he was throwing a bunch of meatballs while also striking out 9 guys per 9. That's usually what happens.

No, its not usually what happens. That's what makes Oswalt, with his big FIP/ERA spread (and even bigger xFIP/ERA) spread, an outlier. But it is a more reasonable explanation for Oswalt being terrible last year than bad luck/bad defense. You can't just dismiss Oswalt's awfulness as random fluctuation.

Either way I have no idea how you generated those comps,

That last list was:

Pitchers with at least 20 starts from 2007-10
With at least 8 K/9
and an ERA+ under 100

Yes, many of them did not pitch much at all from 2011-12. The ones who did mostly did not have good results. This was despite "proving" themselves as ML pitchers with their K/9 rates over 8.

That was the point.

Liriano 2007-2010: 95 ERA+, 8.7/3.5 K/BB, .8 HR/9
Oliver Perez 2007-10: 92 ERA+, 8.4/5.3 K/BB, 1.2 HR/9

Liriano 2011-12: 79 ERA+, 8.6/5.0 K/BB, 1 HR/9

Same pitcher as Ollie Perez.

Jonathan Sanchez 2007-10: 100 ERA+, 9.5/4.6 K/BB, 1 HR/9 (more Ks, more walks, more HR than Liriano 2007-10; slightly better than Liriano 2011-12)

Sanchez 2011-12: 66 ERA+, 8.0/6.5 K/BB, 1.1 HR/9

So to keep making it clear

Paulino 2007-10: 70 ERA+, 8.1/3.9, 1.3 HR/9
Paulino 2011-12: 69 IP

van der Hurk 7-10: 74 ERA+, 8.8/4.6, 1.5
van der Hurk 11-12: 12 IP

Parra 7-10: 81 ERA+, 8.3/4.5, 1.1
Parra 11-12: 59 bad relief innings

Ollie: already covered

Gio 7-10: 95 ERA+, 8.5/4.7, 1.0 (age 22-24 seasons)

Tejeda 7-10: 95 ERA+, 8.4/5.2, 1.0
Tejeda 11-12: 7 IP

Liriano 7-10: 95 ERA+, 8.7/3.5, .8
Liriano 11-12: 79 ERA+, 8.6/5.0 K/BB, 1 HR/9

Villanueva 7-10: 96 ERA+, 8.3/3.4, 1.3
Villanueva 11-12: 104 ERA+, 7.4/3.0, 1.3

de la Rosa 7-10: 97 ERA+, 8.2/4.0, 1.1
de la Rosa 11-12: 70 IP

Nolasco 7-10: 98 ERA+, 8.4/2 (!!), 1.2
Nolasco 11-12: 86 ERA+, 6.2/2.1, .9

Hill 7-10: 98 ERA+, 8.0/4.0, 1.2
Hill 11-12: 28 IP

Kazmir 7-10: 99 ERA+, 8.4/4.1, 1.1
Kazmir 11-12: 1 IP

Sanchez, already covered

So, the question: is Liriano an ML quality pitcher?

Result: using pitchers with similar peripherals and results to Liriano

Paulino, de la Rosa and Kazmir were hurt. Given Liriano's lack of durability and previous injury history, we shouldn't rule this out.

van der Hurk, Parra, Ollie, Tejeda (to my knowledge) and Hill were deemed to not be of ML quality and have bounced between minors and bullpen.

The guys allowed to remain ML starters (at times) for 2011-12 were Liriano, Villanueva (swingman), Nolasco and Sanchez. To repeat, their 2011-12 results:

Sanchez 66 ERA+
Liriano 79 ERA+
Nolasco 86 ERA+
Villanueva 104 ERA+

That's not promising. Of the 12 pitchers in the 2007-10 list (ignoring the young Gio), 3 were hurt, 5 were not ML quality, 2 had terrible results, 1 was a reliable 5th starter, 1 was a successful swingman. I wouldn't like those odds.

Projecting Liriano's 2013-14 we are now looking at Liriano 2009-12:

86 ERA+, 8.7/4.1, .9

These are worse results than Liriano 2007-10 which was followed by two bad years. These results still fit right in with those 2007-10 comps which, as we saw, did not project success.

Here's an interesting historical tidbit. For ages 25-28, for the years 1961-2010, 600+ IP, 8+ K/9, 4+ BB/9, ERA+ <=100 ... there's only one pitcher, Randy Johnson. That's promising. In 2011, Sanchez joined him ... not so promising. With his 2012 season, Liriano becomes the third.

Drop the IP requirement to 300 and you get 10. Names of note added to the list not already discussed are Dave Burba (a late blooming good starter) and Edinson Volquez (84 ERA+ and some injury problems; he'll be 29 in 2013).

Use GS>=20 to pull up a few more AAAA types and you get a list of 13 and none of the new names found any sort of later success.

What that all seems to mean is that pitchers with peripherals similar to Liriano are often not considered to be of ML quality by actual ML teams. Most of the handful of pitchers who have not been hurt and have been given the opportunity to continue starting have not succeeded. Balancing all of that is Randy Johnson (awesome), Burba (damn good for a while) and Villanueva (useful).

Given his recent performance, given the performance of pitchers with similar peripherals and given Liriano's durability issues (about 5.5 IP/start), I see no reason to expect anything better than 150 innings of about 80 ERA+ ball -- i.e. what he's done the last two years. Sounds like borderline AAA/ML to me.

I should add the point about unearned runs is legit. It would be nice if b-r provided RA+ but they don't and I'm not going to figure it out.

I will note that from 2009-12, Liriano allowed 14 UER in 619 IP. From 2007-10, Sanchez allowed only 15 UER in 567 IP (worse but not dramatically so); Volquez 2009-12 only 8 in 404; de la Rosa 06-09 (his age 25-28) 17 in 524; Nolasco 25 in 761; Villanueva just 3 in 338 (maybe a reliever thing) and, a personal fave, Rich Hill 05-08 just 9 in 338.

On the other hand, Ollie did allow 32 in just 483 IP and Parra 27 in 428.

So a low number of UER seems fairly common in this group (no surprise given their TTO outcomes) and it doesn't seem a particularly good indicator of future success/ML opportunity.

So the challenge to the FIPpers is to find high-K, high-BB, FIP<<ERA mismatches who went on to future success along with those that didn't.

Paulino pitched as many innings as Sanchez 2011-2012 (~170), so I'm not sure why you're filing him in the injured list. He had an ERA+ around 100. Ollie Perez isn't a good comp because he had FIP's in the 6's in both 2009 and 2010 - that's the reason why he didn't pitch in baseball (and he actually was good in 30 innings this year).

Here's an interesting historical tidbit. For ages 25-28, for the years 1961-2010, 600+ IP, 8+ K/9, 4+ BB/9, ERA+ <=100 ... there's only one pitcher, Randy Johnson. That's promising. In 2011, Sanchez joined him ... not so promising. With his 2012 season, Liriano becomes the third.

This is the key thing, you have to look at guys who pitched 600+ innings, not guys who pitched 250 innings like Rich Hill (almost all of his career innings came in 2006 and 2007). Sanchez is easily the best comp in the list, but he doesn't really illustrate the original point (about FIP) because his DIPS numbers also deteriorated last year. If Liriano puts up a 6.5 FIP than he clearly is not a major league talent anymore and in the issue becomes moot.

What happens if you look at all pitchers with 600+ IP, 8+ K/9, 4+ BB/9, ERA+ <=100 from ages 24-30 (or some broader range)?

I think 150 innings sounds right. I'd expect an ERA+ of at least 90. I'm guessing ZIPS has him around there when it comes out. Even if you're just doing a 5-4-3 weighted by innings ERA+ projection, you'd get like an 87 ERA+. And we know that A) Liriano has given up far fewer unearned runs than average, and B) his peripherals are far better than that which I know you can't be weighting at zero in the calculation. I don't see how you get him as a AAA starter next year.

Parra should be lumped more with the injured than with the lacking-in-quality

Could it be that Liriano is not a FIP valid MLB pitcher from the stretch but he is otherwise to a degree that overall he is clearly major league talent but one for whom FIP will be not just useless but very misleading? Seems perfectly plausible .

@robert: So in lieu of discussing the issue, you just call people names? And you dare to call someone ELSE a buffoon? You're funny.

FIP is assuming that Liriano's results are randomly distributed. They aren't. He strikes everyone out until someone gets on and then he walks three guys and gives up a home run.

Yeah, that sounds like what I was getting at. I remember thinking something similar of Johnson one his years with the Yankees. He was magnificent with no one on base then would just fall apart when anyone got on in sadly consistent and predictable fashion.

FIP is assuming that Liriano's results are randomly distributed. They aren't. He strikes everyone out until someone gets on and then he walks three guys and gives up a home run.

Right. Since returning from Tommy John surgery, these are Liriano's splits with the bases empty and men on:

235/321/362 (400/175 K/BB, 1688 PA) - Bases Empty
282/362/439 (269/142 K/BB, 1340 PA) - Men On

I'm not saying this absolutely could not be a fluke, and there is a not insignificant BABIP component to the split, but that's a ton of plate appearances distributed over a bunch of seasons. And Twins fans have proposed an entirely reasonable hypothesis for the cause, based on observation. I definitely prefer the "can't pitch out of the stretch" explanation over the "unlucky for five seasons running" explanation.

Liriano has a career 3.42 FIP with the bases empty ad a 4.14 FIP with runners on. That doesn't strike me as an abnormally big split at all considering most pitchers are worse with runners on. Check out the 2012 league split:

http://www.baseball-reference.com/leagues/split.cgi?t=p&lg=MLB&year=2012#bases

I definitely prefer the "can't pitch out of the stretch" explanation over the "unlucky for five seasons running" explanation.

Maybe it's a little of both? Why are people insisting on picking one or the other. We know that pitcher can get unlucky for many seasons and we know that certain things (especially BABIP) are hardly predictive even after half a career:

http://www.baseballprospectus.com/article.php?articleid=14293

If Liriano has an ERA of 4.8 the past 4 years and a FIP of 4.0, it's most likely his true talent was somewhere around 4.4 (or some other number in between).

Also, on FIP/DIPS, I'm seeing one of the early FIP/DIPS mistakes repeated in this thread. It is not the case that "major league quality" pitchers have little variance in hit prevention skill.

In fact, there are highly significant differences between MLB pitchers in their hit prevention abilities. The problem is that this ability is extremely difficult to isolate in samples smaller than 3-5 seasons of pitching. A one-year sample of BABIP is not good evidence of a pitcher's skill in hit prevention. So two things can be true at once - (a) one season of FIP is a better estimate of pitcher skill than one season of ERA, and (b) FIP's assumptions about the lack of variance among MLB pitchers in hit prevention skill are not correct.

Not only do pitchers vary significantly in hit prevention skill, but they vary around a range not dissimilar from the range of pitcher variance in K/9 and BB/9. That sounds crazy, because of course BABIP appears to be highly constrained - even position players pitching keep BABIP around .340 or so, and almost all major league pitchers fall in the .280-.310 band. But BABIP and K/9, when scaled to runs, vary similarly.

Take a league averagish pitcher with a league average 7.5 K/9 in 180 innings. If he improves his K/9 to 8.5, now well above league average, he'll project to prevent about give up about six fewer runs. Now take a league average pitcher with a .295 BABIP in 180 innings. If he improves his BABIP to .285, now well above league average, he'll project to give up about five fewer runs. 10 points of BABIP is worth only slightly less than 1 K/9. That band of roughly 30 BABIP points in which most pitchers fall scales to the roughly 3 K/9 band in which most SP at least fall.

See Allen, Hsu, Tangotiger, "Solving DIPS". And for especially fun reference, see Tango's spreadsheet of regressed hit prevention skill, for pitchers with >500 TBF.

That doesn't strike me as an abnormally big split at all considering most pitchers are worse with runners on. Check out the 2012 league split:

http://www.baseball-reference.com/leagues/split.cgi?t=p&lg=MLB&year=2012#bases

You have to isolate two separate factors here - the first is that pitchers do seem to pitch worse from the stretch, but also worse pitchers allow more baserunners and will be over-represented in the "men on" number, under-represented in the "bases empty" number.

I definitely prefer the "can't pitch out of the stretch" explanation over the "unlucky for five seasons running" explanation.

Maybe it's a little of both?

Yes. That is indeed what I think, and I expressed myself poorly. If we're both saying that Liriano's splits should be regressed toward league average by some rate scaled to PA, but his splits should not simply be replaced with league average splits when making a projection, then we agree.

You have to isolate two separate factors here - the first is that pitchers do seem to pitch worse from the stretch, but also worse pitchers allow more baserunners and will be over-represented in the "men on" number, under-represented in the "bases empty" number.

Good point.

(a) one season of FIP is a better estimate of pitcher skill than one season of ERA, and (b) FIP's assumptions about the lack of variance among MLB pitchers in hit prevention skill are not correct.

Agree with this. Like I've been saying you're always going to weigh K:BB and ERA in some manner. As sample size goes up, you weigh ERA more. It seems like we agree, and I can't think of any other way of interpreting DIPS. The problem is of course what the weights are. Per the link above, even after Liriano's 3500 career PA we'd still regress his BABIP 50% to the mean. And of course there's the problem of him being post surgery. If you just take his past 4 seasons you wanna regress his BABIP like 70% to the mean. I'm guessing runners on splits take less time to stabilize, but I can't find any research on that.

And pitchers with different profiles will strand runners differently.

Totally plausible, but I haven't seen a study which addresses the whole "clutch pitching" issue.

I think there's an aspect of talking past each other going on with JR and everybody else.

FIP absolutely should not be used when it comes to figuring out the value of a pitcher's contribution.

RA+ adjusted for the quality of team defense (and yes, this is tricky. At the extremes, Sid Fernandez and Greg Maddux are going to give up a very different distribution of chances) is the way to go in evaluating the value of a season. Whether or not a pitcher's strand rate is a one year aberration or an actual ability level is of no importance when you're talking value.

Incidentally, I think you'll find that the best projections are an aggregation of the systems mentioned in the thread -- and they still have a pretty big error.

Totally plausible, but I haven't seen a study which addresses the whole "clutch pitching" issue.

Is there somebody who thinks there's no such thing as clutch pitching?

FIP is what the actually pitcher did. ERA is what the pitcher, the team defense, and the team bullpen all did together, as substantially modified by the personal interpretation of not-necessarily-impartial-nor-competent observers.

FIP is part of what the pitcher actually did.

edit: cokes all round.

edit2: "I said that FIP is what the pitcher actually did. True statement."

Nope. It represents part of what the pitcher actually did. Huge differences.

#70 I don't think there's any question that there are clutch results. What's open to question is whether it's an ability, noise or (my educated guess -- I looked at this in a non-systemic way years ago when the data was hard to gather) some combination of the two.

I'm guessing it's about 80% noise/20% actual ability and even at the relatively low signal to noise ratio it's still important.

I don't think there's any question that there are clutch results. What's open to question is whether it's an ability, noise or (my educated guess -- I looked at this in a non-systemic way years ago when the data was hard to gather) some combination of the two.

This is where PitchFX could be useful. A starting pitcher can't dial it up to 98 all game long, nor can he throw a devastating curveball all game long.

Paulino pitched as many innings as Sanchez 2011-2012 (~170), so I'm not sure why you're filing him in the injured list.

a) I probably misread his KC 2011 line as his full line
b) He made only 7 ML and 3 ml starts in 2012 which looks like an injury to me.
c) I wasn't aware Sanchez had been hurt, I assumed he was pulled for sucking but he doesn't seem to have had much ml time

But fine, call Paulino a partial success.

This is the key thing, you have to look at guys who pitched 600+ innings, not guys who pitched 250 innings like Rich Hill (almost all of his career innings came in 2006 and 2007)

Well, no. The question was whether Liriano is a "ML quality pitcher". To determine that you need to compare him to ML and AAAA pitchers to see where he falls. If he is truly "ML quality" he should easily outpitch the AAA guys. He doesn't.

It was further claimed that his 8 K/9 showed he's "ML quality". That list shows this is bollocks.

I found pitchers with similar peripherals to Liriano. As it turned out, I was even lucky enough to find guys with fairly similar UER rates. Most of them continued to do badly.

The only reason to expect Liriano to do better than he has over the last two years is his FIP. I have found pitchers with similar peripherals and similar (some worse, some better) ERA+s as Liriano and shown that they generally continue to post lousy ERA+s or are hurt or get tossed out of the majors. I am asking you to find pitchers with similar peripherals (and age) to Liriano with similar FIP/ERA mismatches who went on to record results in line with their FIP not their ERA.

What happens if you look at all pitchers with 600+ IP, 8+ K/9, 4+ BB/9, ERA+ <=100 from ages 24-30 (or some broader range)?

You tell me, I'm tired of doing all the work when you can't be bothered to try to prove your claim.

And I ####### did this for pitchers aged 25-28 and you get three: Randy Johnson, Jonathan Sanchez and Francisco Liriano. You even quoted this. You really think you're going to get a substantially larger group by expanding the age range?

Don't worry though, see what a nice guy I am, in descending ERA+ order:

dlR
Sanchez
Ollie
Liriano

dlR is the only guy with a post-30 season so far, 11 IP in 2012. So Liriano and possibly Sanchez could pitch their way off the list.

The next best ERA+ on the list is Nomo at 105. He had pretty lousy results at 28 and 29, bounced back to league average, 2 good seasons, then done.

Following him is Dice K at 108. From ages 28-31 he threw 300 innings of 80 ERA+.

JR Richard at 111 -- a) he had a stroke and b) 8 K/9 was a real man in those days.

Ubaldo Jimenez at 112 -- still only 29 in 2013, his 2011-12 is 365 IP of 82 ERA+

Sam McDowell at 112 -- man, this is a depressing list of pitchers ... 29 to 32, 380 IP of 87 ERA+

Finally you get Johnson and Ryan who had turned themselves into real pitchers by age 30 and had ERA+s of 113 and 115.

Top of the heap is Kerry Wood at 120 who kept getting hurt but generally continued to post good results.

It's probably misleading to comp a guy who's only going to be 29 to guys who have already had their 29-30 seasons. If you look at 24-28 instead of 24-30, still 600 IP, you lose some of those guys, Randy Johnson looks like Jonathan Sanchez and you add Park (108) and Langston (109). Park turned to crap and started getting hurt at 29, Langston remained good for several seasons.

So, sorry, high-K, high-BB pitchers with poor performance in their mid-late 20s generally seem to mostly continue to have poor performance -- or cut their BB rate in nearly half and turn into Randy Johnson. Even most of the ones with above-average results seem to have turned into crap or gotten hurt.

You might do better to look at era-adjusted K and BB rates of course but there's no way for me to do that.

The most likely thing of course is that Liriano never fully recovered from his injury.

I don't know if it means anything but I was glancing at his career splits and his H/R splits are massive. An ERA over 1 run higher, K/B over 1 lower, BABIP 25 points higher on the road. The dome and Target both seem to be slight pitchers parks (at a glance) but not to that extent. Does that ring any bells for anybody?

I think 150 innings sounds right. I'd expect an ERA+ of at least 90. I'm guessing ZIPS has him around there when it comes out. Even if you're just doing a 5-4-3 weighted by innings ERA+ projection, you'd get like an 87 ERA+.

Well, 150 IP of 90 ERA+ is not a good pitcher, is borderline ML quality and is not a pitcher you sign for 2/$14. And yes, a 3-year projection will look "good" for Liriano because it includes his strong 2010. In most cases that's fine and always a legit part of the projection. But in Liriano's case, that strong 2010 was preceded by a 76 ERA+ which is right in line with what he did 2011-12. That strong 2010 is also largely a product of an extremely low HR rate which is well out of line with anything before or after and a low BB rate which is completely out of line with 2009, 11 and 12 (but a very high BABIP in 2010). Anyway, 2010 is a pretty big outlier in Liriano's recent performance and I would recommend downweighting it substantially.

And we know that A) Liriano has given up far fewer unearned runs than average,

I covered that! Most of the comps I offered also gave up far fewer unearned runs than average, a couple even less so than Liriano.

B) his peripherals are far better than that which I know you can't be weighting at zero in the calculation.

I comped him to guys with similar peripherals.

I'm not sure you understand the point of the debate. His peripherals are only "better" if FIP is a good model for guys like Liriano. If high-K, high-BB pitchers who post lousy ERAs continue to post lousy ERAs (or get hurt or are out of the majors) then there is no justification to expect Liriano to pitch to his FIP.

Show me high-K, high-BB pitchers of a similar age with ERA substantially worse than FIP who then went on to pitch to their FIP. I'd be especially interested in ones who did so without lowering the BB rate substantially. The reason Randy Johnson became a great pitcher is because he doubled his K/BB (while K'ing 3-4 more per 9 than Liriano). If Liriano can post 2010 K and BB numbers, he'll be a much better pitcher -- he could become Javier Vazquez! :-)

But if all we're arguing over is 150 IP of 80 ERA+ vs 150 IP of 90 ERA+ then what are we arguing over? If FIP is saying he's a solid 5th starter and ERA+ is saying he's a lousy 5th starter -- that's all within prediction error. Still not a guy you sign for 2/$14.

Walt -- a little difficult to read, but interesting stuff. Thanks for looking it all up. You've got me mostly convinced (except I don't think the comparison to Oliver Perez holds, as I don't think you can hand-wave away a discrepancy of ~1.5 BB/9)

Liriano has a career 3.42 FIP with the bases empty ad a 4.14 FIP with runners on

Can't use career numbers. He was great until a couple years ago and lost control of his slider from the stretch. Use the last 2 seasons unless you think he's going to return to form.

Target Field is a pitcher's park that especially favors LHP, just like Metrodome. Deep LF and really deep to the power alleys.

I remain convinced that ERA is a garbage dinosaur stat and a poor pitching metric, and that we are all better served by using FIP instead.