There’s a debate raging within these “walls” that has plagued evaluations of pitchers from the very beginnings of the Hall of Merit: how to assess the value of relief pitchers in general, and closers in particular.

Allow me, if I may, to take a stab at defining the parameters of this debate. These are my personal opinions, and are based in large part on my understanding of the history of the game. Comments, extensions, criticisms and corrections are explicitly invited.

This is NOT a thread for “Relievers are great” vs. “Relievers suck” flame wars, but rather an intended home for comment and systematic analysis centered on the correct evaluation of the value of relief pitchers in general, and closers in particular, in the context of the election criteria of the Hall of Merit.

Part 1: Rationale

In his brilliant post #485 in the 2019 ballot thread, @Dr. Chaleeko puts his finger on the pulse of the philosophical divergence underlying this debate:

I think the word "position" has been a stumbling block. I suspect, Prog, based on your posts, that you are not necessarily advocating for a change in the definition of position as I've talked about it above, but rather an informal definition within the context of each voter's deliberations. As in "We should, as voters, consider closer as a position as we analyze players." Not as in, "We should turn back 150 years of common usage and officially define closer as a position."

Absolutely correct.

I believe a ‘closer’ can be defined, quite simply, as a pitcher who finishes a high percentage of his team’s games. Just as the definition (as opposed to the success) of a ‘starting pitcher’ has nothing to do with wins or rate stats, but rather with games started, the definition of a ‘closer’ has nothing to do with saves or rate stats, but rather with games finished.

That part is easy. It’s wildly differing perceptions of the value of a closer – and even the legitimacy of the term itself – that roil the electorate.

The nature of pitching has been in flux since the beginnings of organized baseball – more so, by far, than the nature of any other position. There have been numerous changes to the pitcher’s role and the conditions under which pitchers ply their craft, including:

* Changes to the distance from the mound to the plate
* Changes in the meaning of balls and strikes
* The shift from underhand to overhand pitching
* Visual neutralisation of the hitters’ background behind the pitcher through the introduction of batter’s eye screens
* Elimination of amateurs from MLB play
* The introduction of cork-centered baseballs
* The banning of doctored pitches (which, together with:)
* The immediate replacement of scuffed balls (helped create the conditions for:)
* Babe Ruth and the advent of the live ball (which in turn helped bring about:)
* The decline of the stolen base
* Systematization of the minor leagues
* Use of lights in night baseball
* Integration, and the rebirth of the aggressive running game
* Amphetamine use
* Lowering of the mound after 1968 (I call this the Bob Gibson Effect)
* The designated hitter and the demise of the double switch in the AL
* Videotape replay, and the consequent advance of hitters’ understanding of pitchers’ tendencies and vice-versa
* The emergence of carer relievers and the decline of the complete game and shutout
* Bill James and the rise of systematic mathematical analysis of statistics
* The evolution of the forkball into the more effective split-fingered fastball
* The radar gun
* The Steroid Era
* Personal trainers, customized dietary regiments, and laser eye surgery
* 3-D video mapping of pitch trajectories
* The decline of the sacrifice bunt and the low-percentage steal
* Increasing acceptance of high strikeout totals for productive hitters
* Near-universal adherence to pitch counts and the concomitant surge in dependence on game-finishing relievers (i.e., closers)
* The increasing prevalence of defensive shifts
* The first tentative use of openers

There are more to be sure, but this list will suffice to illustrate the central point: Despite its long history, and the veneration with which we regard the greats of years gone by, baseball is not a static institution fixed in the past, and cannot be analyzed as though it were. Baseball has been in a state of constant flux from Abner Doubleday till now. It’s one of the many factors that make the game so alive to modern fans like ourselves.

(And still, after all the intervening decades and evolutionary steps, the throw to first beats the batter by half a step. What a game!)

Of the many changes that have swept baseball since the Major Leagues were thoroughly professionalized circa 1905-1910, the ones that have resulted in by far the most radical alterations to the definition of the term ‘pitcher’ are:

* The advent of the live ball, and
* The emergence, and still-continuous re-definition, of the closer.

The live ball transformed the nature of pitching. The ban on doctored pitches left pitchers with far less control over the spin and motion of the baseball; the constant availability of clean, high-visibility balls tilted the odds even further in the batters’ favor.

It’s important to note, in the context of the present debate, that the nature of starters’ statistics altered massively at this time. Scoring and long hits soared while strikeouts plummeted. Pitchers could no longer pace themselves as they had in the past; they needed to throw much harder, and throw hard much more constantly, to compensate for their reduced ability to affect the motion of the ball. The number of games starters could pitch in a season, and the number they could complete, fell dramatically.

No one expected any post-1920 pitcher to challenge Cy Young’s 511 wins, 7356 innings pitched, or 749 complete games – and yet we speak of a number of post-1920 pitchers as being qualitatively superior to Young. His stratospheric career totals are recognized as not making Young “better than” or “superior to” modern starters – because it’s also recognized that the nature of pitching itself, and the meaning of the word ‘starter’, changed radically after 1920. (One of the factors that makes Walter Johnson shine so brightly among his contemporaries is that he navigated this change successfully and retained his status as a dominant starter – not as dominant as he was prior to 1920, but still good enough to take home the 1924 AL MVP.)

The second major change to the nature of pitching is the emergence, and constant re-definition, of the career reliever, and the advent of the closer in particular. Once again starters’ statistics are undergoing a fundamental redefinition – a change that is far from complete as I write these words. 30-win and 15-complete game seasons are relics of the distant past. Jacob deGrom won (and deserved, in my view) the 2018 NL Cy Young Award with just 10 wins – an outcome impossible to envisage just ten years ago. His 217 innings pitched and one (1!) complete game pale in comparison to the dominant starters of a single generation past – yet we do not speak of him as being less valuable or less successful than his predecessors.

I hold that this ongoing evolution is even more radical in the totality of its impact on the nature of pitching than the advent of the live ball – and therefore merits a thorough re-working of the way we think about and analyse pitching in general, and relief pitching in particular.

Part 2: Analysis and Definition

Please forgive me, Doctor, for abbreviating your words in the following quotation; I’m not trying to cherry-pick your logic, I swear. The full quote is available in your original post; herein, I’m looking to cut straight to the meat of the debate.

Position is specific terminology that has a 150 or so year tradition of common usage referring to the place where, generally, someone stands on the field while on defense […] In this frame of reference, all pitchers are just pitchers […] They accumulate almost all the same stats.

The first two assertions are definitely correct. (And very perceptive too, as they cut through a lot of the fog surrounding this debate). The final assertion, however – that pitchers accumulate “almost all the same stats” – is where we run into problems.

Relievers accumulate a different set of stats than starters. They do accumulate saves and holds (which lack analytical substance), and also games finished (germane to the definition of ‘closer’, less germane as a basis for evaluation). They do not accumulate games started, complete games, or shutouts, and are limited, by the nature of their usage, in the accumulation of traditional counting stats. Relievers will also tend not to win titles in traditional rate stats for which there are minimum innings requirements.

So, if the stats that relievers do accumulate tend to be of limited analytical value, we need to define the differences between the two groups in different terms.

Here follows a series of qualifying statements intended to narrow and sharpen, and thereby explicitly define, the differences between starters and relievers.

* The purpose of relief pitching can be defined as consistency of high-level performance in short-duration, high-leverage situations.

* The definition of success for a relief appearance is not an individual display of brilliance, but rather the maintenance of stasis in a game, i.e., the preservation of winning and tied situations.

* In light of the preceding definitions, it is inherent to the nature of the role that relievers will generally surpass starters in ERA, ERA+, WHIP, and strikeout-to-walk ratio. However:

* Inherited runners who do score are not credited to a reliever’s ERA. This must be taken into account as one of the factors holding down relievers’ ERA comparative to starters. (WHIP is not affected by this paradigm.)

* Since the 1970s:
--> Starters’ innings pitched and games finished are declining.
--> Relievers’ games finished are increasing.
--> Relievers’ overall share of innings pitched has increased – but individual relievers’ shares of innings have declined.

* Starters and relievers will amass substantially different statistics:
--> Starters will generally surpass relievers in innings pitched.
--> Relievers will generally surpass starters in number of appearances.
--> Starters will generally surpass relievers in accumulation of WAR and Win Shares.
--> Relievers will rarely if ever lead a league in a non-relief-specific statistical category, either counting or rate.

* Numbers of appearances and games finished, while of limited analytical value, are nonetheless better indicators of a reliever’s value to a team than innings pitched, wins, saves, or holds. Any theoretical HoM baseline for relievers’ careers should definitely be somewhere below 1000 innings pitched.

* Because relievers generally pitch in higher-leverage situations than starters, they will generally surpass starters in rate stats that reflect per-inning leverage. Despite this:

* Irrespective of how statistics are concatenated, the raw disparity in innings pitched ensures that starters will virtually always surpass relievers in total game-winning value – however one chooses to define that term.

* Relievers enjoy various competitive advantages over starters, including:
--> Pitching fewer innings – and thus, because they don’t have to pace themselves to the same extent:
--> Being able to throw with greater per-pitch velocity over the course of their appearance;
--> Seeing batters less often, within both a game and a season;
--> Being able to observe at-bats prior to their entry into the game, thus giving them more evidence on the batters than the batters have on them; and
--> Generally seeing each batter only once per game, and thus being able to rely on a limited vocabulary of pitches comparative to starters.

* Relievers often enter a game with runners on base. It’s therefore especially important for relievers, who generally throw at above-average velocities, to nonetheless be able to induce ground balls in key situations, in order to prevent sacrifice flies, promote threat-ending double plays, and enable plays at the plate. This is much more important to a reliever’s success than the ability to strike out a given batter – which means in turn that pure power pitchers will, as a rule, be less successful in a relief role than pitchers who can finesse batters.

* Closers (as distinct from relievers in general) will successfully protect winning situations at a much higher rate of success than starters can create winning situations. For example: Greg Maddux maintained a steady value as a starter in his declining seasons, because he ate innings at a predictable rate of value, even though that rate was sharply decreased from his younger years. A closer who shows a similar decline in rate stats will quickly lose his designation and be demoted to a utility role. (This is the #1 reason why Mariano Rivera was uniquely valuable: he protected winning situations with historically high and 100% predictable rate stats for an unprecedented number of consecutive years.)

* Endurance and usage patterns being more or less equal, relievers with approximately equivalent rate stats will protect winning and tied situations at an approximately equivalent percentage of success, and therefore offer approximately equivalent value to their teams. This makes the majority of relievers, as individuals, more or less “generic”.

* The above condition does not diminish the value of relievers as a class. Teams will always find a high-leverage use for pitchers with limited endurance but excellent rate stats – especially as the role of starting pitchers continues to evolve, and relievers (and possibly openers) consume an ever-larger share of team innings.

* Because relievers’ value is based on their reliability in high-leverage situations, they must perform near, at, or above their regular-season level in post-season situations to have substantial value to their teams. (Otherwise they’re a bit like the NFL’s Gary Anderson in 1998: he was perfect on every field goal and extra-point attempt in the regular season – the only kicker in history to achieve this – but muffed an easy game-clinching kick in the NFC Championship Game. No Vikings fan alive would describe Anderson’s record-setting season in positive terms, because he blew the highest-leverage opportunity of the entire year, the one that would have put his team in the Super Bowl. And so it is with relievers.)

*Whether a reliever earns his role through failure as a starter, specific grooming for the role, or through any other circumstances, is 100% irrelevant to the value he provides to his team. So long as he can induce ground balls in key situations, whether he commands a single pitch or an array of pitches is equally irrelevant.

* As starters hew more closely to absolutist pitch counts and pitch less deeply into games, it may well be that middle relievers with good endurance will attain a newly enhanced importance in baseball. Firpo Marberry might prove to be a template for an entirely new class of pitcher that will dominate the middle innings of games in future.

Part 3: Evaluation

Taking key points from the above into account, let’s take a stab at defining what constitutes a great reliever. (Hint: It has nothing to do with saves.)

For a reliever to attain excellence sufficient to legitimately merit enshrinement in the Hall of Merit / Fame, I believe they must comfortably meet the following criteria:

1. Rate stats considerably superior to those of league-average starters. (Lee Smith falls well short on this standard.)

2. Large numbers of leveraged appearances per year – or, in the case of 1970s relievers (and, perhaps, of a new generation of middle relievers), substantial numbers of leveraged multi-inning appearances.

3. Very consistent success in protecting winning and tied situations. (The Save and Hold statistics do not adequately document this condition.)

4. Rock-solid consistency over a long and consecutive multi-year stretch. (I can’t imagine less than a full decade being sufficient.)

5. Consistency in postseason play at least approximating – and preferably equal or superior to – their regular season play. Postseason series hinge on a very small number of wins and losses comparative to the regular season. Regular-season reliability in preserving wins means little if that reliability fades as the necessity for each win – and, therefore, the leverage inherent in each appearance – increases. (This is where Billy Wagner misses out.)

It may be observed that few relievers historically can meet these standards. That’s as I intend it to be. In my view, for a reliever to be taken seriously as a provider of sufficient game-winning value to merit Hall enshrinement, they need to be exceptionally dependable at their job, regular- and post-season, for a very long period of time.

That’s my take. I await your views. Thanks for reading.

I have a few nits to pick with your analysis in part 3:

1. Successful relief pitching is not just in protecting winning and tied situations. You give no credit to a pitcher coming with his team losing by 1-2 runs and keeping the score close enough for his team to come back and win. You also rate a pitcher coming in with a 5 run lead and giving up 4 runs as successful.

2. Are you really implying that a relief pitcher that pitches for bad teams (e.g. no or minimal post season pitching) is excluded from the HOF?

3. Rate stats considerably superior to league-average starters (from memory ~95 ERA+) is a pretty weak criteria. Shouldn't this be considerably superior to HOF level starters?

4. You don't have any criteria on allowing inherited runners to score. How well the pitcher does in these situations don't show up in the rate stats. This makes an unequal comparison between middle relievers (e.g. Miller) and closers. Middle relievers often come in with runners on base while closers almost always start innings. This penalizes middle relievers since they have a much harder job in protecting winning and tied situations.

Evaluating players I first look at how many players did something similar or better than each of the player's HoF arguments. For me with relievers it will be usually either how many IP in seasons reaching ERA+/OPS+ benchmarks or how many IP in a consecutive run of seasons by ERA+/OPS+.

Mark Eichhorn 1986-94: 817.1 IP, 150 ERA+, 77 OPS+
Brad Ziegler 2008-18: 717.1 IP, 149 EA+, 81 OPS+
Kyle Hendricks 2014-18: 789 IP, 134 ERA+, 77 OPS+

I don't think of any of them as having a HoF quality career/prime so far. The main issue with evaluating relievers is if you consider Ziegler and Hendricks to have significantly different career value to this point. Hendricks has the drawback of facing the same batter 3 times in an outing and the advantage of only having to maintain that level of performance through Age 28 and now has another 10 years to vastly surpass Ziegler's value. If you consider Ziegler considerably more worthy or vice versa then restrict comparisons based on the pitcher's role and best of luck handling hybrids.

309.2 IP for Eichhorn in seasons (1986, 91, 94) with 200 ERA+ (or 208) and 60 or lower OPS+ (or 55) which is really close to Billy Wagner 307.2 or 323.1 if you don't set a min IP per season with a 247 ERA+ and 41 or lower OPS+ but no other season meeting the lesser criteria. 1978 Ron Guidry 273.2 IP, 208 ERA+, 50 OPS+.

Taking out those/that season(s) Eichhorn is a mediocre reliever, Wagner is still an excellent reliever and Guidry has another 6 seasons where he's a top tier starting pitcher. Wagner is very often successful in his 307 appearances (incl the 15.2 IP season) and Guidry's season was very much needed to reach Game 163 and has a solid outing in Game 163, leaving with 1 out and a runner on 1st with the Yankees leading 4-2 in the bottom of the 7th.

Gossage shuts down the Red Sox that inning but his entire outing is 8 outs, 2 runs and 0.222 WPA while Guidry is 19 outs, 2 runs and 0.057 WPA, Gossage is nearly 4x as "worthy" by that metric as it's clear that every run matters late in the game. If you consider WPA a relevant stat or leverage adjusted WAR or Win Shares or xFIP or whatever compare using those stats, some of the stats will essentially segregate starters and relievers.

If you want to compare relievers to position players keep in mind that a 90 OPS+ pitcher and 120 OPS+ hitter are both 10% better than league average.

Mariano Rivera: 5103 PA (BF), 208 OPS+ (46)
Sandy Koufax 1963-66: 4651 PA, 188 OPS+ (56)
Mike Trout: 4673 PA, 175 OPS+
Billy Wagner: 3600 PA, 202 OPS+ (49)
Barry Bonds 1999-05: 3536 PA, 229 OPS+

@Cleveland (need a new name) fan:

Thanks for joining the debate! You raise some valid and perceptive criticisms.

1. Successful relief pitching is not just in protecting winning and tied situations. You give no credit to a pitcher coming with his team losing by 1-2 runs and keeping the score close enough for his team to come back and win.

I agree with this. A better phrasing might be something analogous to, “preservation of the existing game state”. I’ll give this some thought.

You also rate a pitcher coming in with a 5 run lead and giving up 4 runs as successful.

Well, no, because a pitcher like that would have lousy rate stats. That’s why superior rate stats are my #1 criteria. But again, “preservation of the existing game state”, or something akin, covers that contingency better than my original wording.

2. Are you really implying that a relief pitcher that pitches for bad teams (e.g. no or minimal post season pitching) is excluded from the HOF?

No, I really am not. I said that postseason pitching must be equivalent to or better than regular season pitching. This does not say, or imply, that a pitcher must have post-season experience.

However, if you find the working unclear, then others might as well. A qualifier might be in order.

3. Rate stats considerably superior to league-average starters (from memory ~95 ERA+) is a pretty weak criteria. Shouldn't this be considerably superior to HOF level starters?

Yeah, I think I agree with that.

4. You don't have any criteria on allowing inherited runners to score. How well the pitcher does in these situations don't show up in the rate stats. This makes an unequal comparison between middle relievers (e.g. Miller) and closers. Middle relievers often come in with runners on base while closers almost always start innings. This penalizes middle relievers since they have a much harder job in protecting winning and tied situations.

Oh, I agree. I mentioned above that inherited runners do not impact relievers’ ERA, and that this has to be taken into account as a factor in suppressing relievers’ ERA relative to starters. This is actually a rate stat advantage for middle relievers relative to closers. This is also why I specified that saves and holds have little evaluative value, and why my criteria for evaluation are based primarily on conditional statements rather than statistical benchmarks – because the statistical benchmarks just aren’t there for substantive analysis.

I encourage posters (including yourself) to submit ideas for the calculation of actual crunchable numbers. We’re working in areas of debate that have yet to be resolved to anyone’s satisfaction. The more ideas and voices, the better.

Thanks for your answers.

Well, no, because a pitcher like that would have lousy rate stats. That’s why superior rate stats are my #1 criteria. But again, “preservation of the existing game state”, or something akin, covers that contingency better than my original wording.

If you are going to defer to rate stats, why do you need this criteria at all? Within this criteria, giving up 4 runs with a 5 run lead is a success.

said that postseason pitching must be equivalent to or better than regular season pitching. This does not say, or imply, that a pitcher must have post-season experience.

I see where I was misreading it. It still might be a good idea to make it a little clearer however.

This is actually a rate stat advantage for middle relievers relative to closers.

I don't understand your point at all. Since inherited runners scoring has no effect on a relievers rate stats, how does this create an advantage for MR? Maybe a relief pitchers rate stats should be dinged for allowing inherited runner to score, but they aren't today. Also coming up with a good criteria for dinging pitchers is hard since it is so situation dependent. A runner on 1st with 2 outs is a lot different than runners on 2nd and 3rd with no outs. There is a lot more blame in allowing one run to score in the first situation than the second, although the results are the same. Also if a pitcher should be dinged for allowing inherited runners to score, should they get a rate bonus if they strand inherited runners?

On the other side, your criteria explicitly penalizes MR. You have explicit criteria for keeping the lead or a tie which is more difficult when you come in with runners on base versus starting an inning. Also, although I am too lazy to look it up, isn't offense higher with runners on base relative to nobody on base (e.g., 1B have to hold runners, pitchers have to slide step etc.) which penalizes MR's rate stats.

--------------
New thoughts

Rate stats take park effects into consideration, but your other criteria do not. However, criteria like number 3 are directly affected by the park. It is easier to keep a game tied or maintain a small lead in San Diego than Colorado, which has the effect of giving someone like Hoffman an advantage over someone who pitched their whole career in Colorado.

When do SSS negate your criteria? Wagner pitcher 11 innings in the post season. He was terrible in those few innings, but are they meaningful for a HOF case. There are a lot of reasons beyond his skill that might account for this lousy performance. He might have been nursing a small injury or he might have been gassed because he was overpitched late in the season to make the playoffs etc. Should there be a minimum number of innings before criteria like number 5 start to come into play?

* Relievers often enter a game with runners on base. It’s therefore especially important for relievers, who generally throw at above-average velocities, to nonetheless be able to induce ground balls in key situations, in order to prevent sacrifice flies, promote threat-ending double plays, and enable plays at the plate. This is much more important to a reliever’s success than the ability to strike out a given batter – which means in turn that pure power pitchers will, as a rule, be less successful in a relief role than pitchers who can finesse batters.

Isn't this the exact opposite of what we see actually happening in MLB?

Isn't this the exact opposite of what we see actually happening in MLB?

It certainly is trending that way, yes.

Then again, I don't see anyone in MLB right now who I'd consider electable. Perhaps the two conditions are not unrelated...? I have no empirical basis for that supposition - but I'd sure love to see stats on how many of today's relievers try to blow the ball past the batters with runners on and less than two outs. I've personally seen Chapman, for one, be guilty of that, and pay a nasty price.

I hold there's more to being a great reliever than being able to blow the ball past the batters.

I encourage posters (including yourself) to submit ideas for the calculation of actual crunchable numbers.

I would humbly suggest that my Player won-lost records are a good tool for analyzing relief pitchers. The core of the system is Win Probability, expressed as "wins" and "losses" with two improvements. First, I credit wins and losses to baserunners and fielders in addition to batters and pitchers. And second, I make a final adjustment so that all games are valued equally.

I made adjustments recently to two of the calculations that I use to compare players, both of which affect relief pitchers quite a bit. See here. I have changed my positional averages so that all pitchers are measured against the same average (0.500). And I have changed my eWins so that they use actual context, so that total eDecisions for a player (eWins + eLosses) equal that player's pDecisions. A player's eWins may still differ from his pWins, however, depending on the timing of his performance.

I wrote an article comparing starting pitchers to relief pitchers a few years ago, which I just updated a bit. See here. I also looked at the best relief pitchers in an article I wrote last month about the newest Hall-of-Fame reliever, Lee Smith.

As for relief pitchers in the Hall of Merit (and the Hall of Fame, I guess), the problem I run into is what I'd call "the Mariano Rivera problem". Mariano Rivera is a deserving HOF/HOM member. You can construct a strict statistical argument against him, but if you give him postseason credit, credit him for the context in which he pitched (i.e., he pitched mostly high-leverage innings), and compare him against a high enough baseline (he looks better vs. average than vs. replacement level; his raw pWins are quite low for a serious HOF/HOM candidate), you can get him to pop up pretty high on a HOF/HOM ballot without making any particularly unusual adjustments or treating relief pitchers as a special case. For example, if I give eWins twice the credit of pWins and give equal weight to wins, WOPA, and WORL, with no extra weighting for relief pitchers - none of which is particularly favorable to Rivera, Rivera pops up #62 in my UberStat (between Bobby Grich and Red Ruffing) - he's basically surrounded by HOMers. Meanwhile, the second-best relief pitcher to show up is Goose Gossage, who shows up at #284 (since 1921) which is almost certainly outside of my personal Hall of Fame.

But if you give relief pitchers a boost to get another reliever or two into, say, the top 250, you'd have to "goose" Gossage's numbers (yeah, the pun was intended, sorry) by 6.5% or so. Which doesn't seem like a lot, but if you also boost Rivera's numbers by 6.5%, Rivera moves up from #62 to #48.

I'm not entirely comfortable with the idea of Mariano Rivera as one of the top 100 players of the past century. Pushing him up to one of the top 50 players of the last 98 years seems way too far. But if Mariano's not one of the top 50 players of the last 100 years, then, by my numbers, he's the only relief pitcher good enough for the Hall of Merit (Dennis Eckersley and John Smoltz are also there, but in both of their cases, a majority of their value came from starting - Smoltz, in particular, is in my top-100 even if you zero out his relief pitching entirely).

Basically, Mariano Rivera proved that it is possible for a relief pitcher to amass a Hall-of-Merit worthy career without having to resort to gimmickry in one's treatment of relief pitchers. But he was SOOOO much better than any other relief pitcher that his existence basically cuts the legs out from under everybody else's case.

Anyway, those are my two or three or four cents on relief pitchers and the Hall of Merit.

another advantage that closers have over other Ps is that they have a "suck cap."

that is, enter with a 1-run lead in the 9th, allow single, single, walk, walk - and you allow 2 ER in 0 IP and lose. but if anyone else does that - say, a SP in the 6th, or a setup man in the 8th - their misery is not yet over. the bases are loaded with none out, and 1, 2, or 3 ER may be added to your tally even as you depart to the taunts of the home crowd.

but for the closer, it's 2 ER - thus the "suck" has a "cap."

but for the closer, it's 2 ER - thus the "suck" has a "cap."

Maybe "closer suck" should be weighted more heavily than "MRP suck" or "SP suck". Also you should factor in what part of the lineup the closer faces. Succeeding against the heart of the order should be recognized. Failing against the bottom half of the order should be recognized.

Stepping back a bit, we've elected Gossage, Wilhelm, and Rivera easily. We also inducted Rollie Fingers by a narrow margin. Bruce Sutter, Lee Smith, and Trevor Hoffman are honored in Cooperstown but not in the Hall of Merit; only Smith receives meaningful support here.

I supported the induction of Fingers in real time, regretted it as a mistake, and have come around to support it again. No one except Fingers held up under '70s fireman workloads for a decade, not even Gossage, who had one 100-IP season after age 26 - only 102.1 innings, at that. In Fingers's 12-year prime (1971-82), he exceeded 100 innings nine times. The exceptions were 1979, the only "down" year in the mix; 1981, the strike year; and 1982, when he was on pace for 100+ when he suffered an arm injury and was never the same.

Gossage is a peak, not a prime, pitcher. He had three otherworldly years (1975; 1977-78), seven years when he was insanely effective but limited in his usage, like a modern closer (1979-85), and his league-average [for a reliever] late career phase.

#13...YES!

I don't recollect that WPA and other reliever stats take quality of opposition hitters into account. I don't think that WAR does either. But this is kind of a big deal. Here's Mariano Rivera's PAs against each slot in the order. [Note: I didn't pull out his 10 GS.]
1st: 540 (10.6%: .477 OPS against)
2nd: 517 (10.1%: .531 OPS against)
3rd: 506 (9.9%: .723 OPS against)
4th: 536 (10.5%: .717 OPS against)
5th: 580 (11.4%: .548 OPS against)
6th: 612 (12.0%: .498 OPS against)
7th: 617 (12.1%: .508 OPS against)
8th: 624 (12.2%: .498 OPS against)
9th: 571 (11.2%: .533 OPS against)
TOTAL: 5103 PA (.555 OPS against)

So the two toughest outs he would face, the only two who hit him near the league average, were the guys he faced the least and the second to least. Put another way, the three slots he faced the most averaged a .501 OPS against him, and the guys he faced the least averaged .657. In his own era that's the difference between facing Livan Hernandez, a good hitting pitcher, and Joe McEwing. Or another way: It's the same difference in OPS as Lance Berkman and Jeffrey Hammonds.

Clearly in a given season, it's not much more than 5 to 10 PA, but, boy, doesn't the save situation look a lot easier if I'm facing two or three Livan Hernandezes?

Of course, a SP's version of that little table simply declines in percentages slot by slot because they enter in the first. So they are facing the good hitters more often per game.

I also wanted to see how the difference in batter quality interacted with other traits of relieving. So I just picked the 2005 AL because Mariano had a great year that year.

Looking at AL-wide splits:
SP ERA - RP ERA = 0.49
In 2005 AL, RP gained a half-run advantage per nine innings in ERA by dint of the difference in difficulty of their roles. This is the same difference as 2005 Mark Buehrle and 2005 Jose Contreras.

SP OPS - RP OPS = 0.034
In 2005 AL, RP gained a 0.034 advantage per innings in OPS by dint of the difference in difficulty of their roles. This is equivalent to facing 2005 Jason Varitek instead of 2005 Gary Sheffield.

ERA in 1st inning of game - ERA in 2nd inning of game: 0.83
The first inning of a game for the SP is the only time when the offense gets to line up its hitters the way it wants to against the opposing pitcher. It confers an advantage to the offense of nearly a full point of ERA, or the same difference in 2005 as Johan Santana and Josh Towers.

OPS in 1st inning of game - OPS in 2nd inning of game: 0.55
Same deal as with ERA. The difference in hitters the SP will face in HIS first inning is the same as the difference between Johnny Damon and Brandon Inge c. 2005.

I have no great conclusion other than the quality of hitters facing a RP probably has a very big impact.

that is, enter with a 1-run lead in the 9th, allow single, single, walk, walk - and you allow 2 ER in 0 IP and lose. but if anyone else does that - say, a SP in the 6th, or a setup man in the 8th - their misery is not yet over. the bases are loaded with none out, and 1, 2, or 3 ER may be added to your tally even as you depart to the taunts of the home crowd.

but for the closer, it's 2 ER - thus the "suck" has a "cap."

Only on the road. And while it doesn't offset the suck cap, they also have an outs cap. If they yield a two-run homer to lose the game, they don't get the run-free out that the seventh-inning reliever would usually follow with.

The contributions here are fantastic. This is exactly why I wanted a dedicated thread to discuss these issues.

Crucially, the analytical side is, I think, beginning to come into focus.

Some proposals on quantification of the analytical aspect:

* Relievers’ overall success in run suppression could be evaluated not by ERA, which fails to penalize relievers for allowing inherited runners to score, but by a new stat that tallies earned and inherited runs allowed by a reliever. I propose the acronym RRA (Relief Run Average).

* Relievers’ success in preserving game state could be evaluated not by saves or holds, but by a new stat that counts how many appearances a reliever makes, irrespective of length, in which he allows no earned or inherited runs. I propose the acronym ShRA (Shutout Relief Appearances).

* ShRA could be presented as a percentage stat – roughly analogous in its purpose to winning percentage, but far more germane to actual performance analysis – by dividing ShRA by total relief appearances, figured to three significant decimal places. I propose the acronym ShRA%.

* Minimum benchmarks for seasonal leaderboards in the above stats could be set at 50 games (slightly less than 1 appearance per 3 games) or 80 innings (slightly less than 1 inning per 2 games) – levels that regularly-used middle relievers, setup men and closers should all comfortably surpass.

Calculating the above stats for the entirety of the relief era would, I suspect, show strong observable trends from year to year. I think you’d likely see the same names in the leaderboards year after year.

* * *

More notes.

@Kiko Sakata:

I'm not entirely comfortable with the idea of Mariano Rivera as one of the top 100 players of the past century. Pushing him up to one of the top 50 players of the last 98 years seems way too far.

Agreed. But change “players” to “pitchers” – a much lower benchmark – and I personally could accommodate him on either count.

The problem I run into is what I'd call "the Mariano Rivera problem" […] He was SOOOO much better than any other relief pitcher that his existence basically cuts the legs out from under everybody else's case.

Absolutely correct. Mariano sets the upper end of the argument very, very high. And I for one don’t have a problem with that. The lower end shouldn’t be all that much lower, I don’t think, if we are to acknowledge the central fact that relievers simply don’t accumulate value at the same rate as starters.


@Dr. Chaleeko:

Fascinating stuff on performance vs. batting-order slot; may I ask from whence you obtained the data?

The quality of hitters facing a RP probably has a very big impact.

One possible approach to rendering this information usable for analysis is to calculate the differential between relievers’ actual performance and league-average performances vs. each slot in the batting order. Plotted as an overlaid frequency polygon, with batting-order slot as the X axis and actual vs. league-average OPS as twin Y axes, the results should show roughly parallel lines for a great reliever, who should achieve approximately similar levels of suppression of opponents’ OPS at all levels of opponents’ offense.

Not sure how this could be translated into a baseline stat though; any thoughts?

The problem I run into is what I'd call "the Mariano Rivera problem" […] He was SOOOO much better than any other relief pitcher that his existence basically cuts the legs out from under everybody else's case.

his departure left the Yankees to muddle through with these closers in the years ahead:

2014 - David Robertson 39 SV
2015 - Andrew Miller 36 SV
2016 - Aroldis Chapman+Dillon Betances 32 SV
2017 - Aroldis Chapman+Dillon Betances 32 SV
2018 - Aroldis Chapman 32 SV

is anyone saying that the Yankees would have had much of a better W-L record with vintage Mariano?
closers save a high percentage of their games.

again, this ain't figure skating or diving, where a slight flaw in a performance changes your score.

DID YOU SAVE THE GAME? YES
THANK YOU, WE'RE DONE HERE

DID YOU SAVE THE GAME? YES
THANK YOU, WE'RE DONE HERE

Boy, do I NOT see that statement - or indeed the save stat itself - as how to evaluate a reliever's performance.

Is anyone saying that the Yankees would have had much of a better W-L record with vintage Mariano?

The Yankees won 13 pennants with Mariano as their closer, or 72% of all possible seasons, versus zero for five in the time frame you cite.

The Mariano-less Yankees also went 1 for 3 in playoff series (1 for 4 if you count the 2015 wild card), with Chapman personally helping his team to bomb out of the 2017 ALCS.

So perhaps not a great comparision with the Rivera-era Yankees.

I have no great conclusion other than the quality of hitters facing a RP probably has a very big impact.

Thank you for doing the heavy lifting for me. :-)


It's a ###### up system when a guy who only has to go 1 inning gets paid more than the middle reliever has to go 2 or sometimes more. I believe in paying by the inning, IOW, starters get the big money, followed by the RP inning eaters. I understand the perceived need for a closer, but I just have come to hate the idea of it. Jonathan Papelbon was loved, for a while, here in Boston, and he really wasn't very good if he had to throw more than 20 pitches in a game. He was a failed starter, as just about all relievers are, and for that reason, they don't belong in the HoF. When a guy is good enough to be a #4 or 5 starter for his entire career and has mediocre stats and gets no HoF consideration, but the guy who failed as a starter and gets the opportunity to go all out for 60 innings a year gets the glory...there's something unfair here.

"The Mariano-less Yankees also went 1 for 3 in playoff series (1 for 4 if you count the 2015 wild card), with Chapman personally helping his team to bomb out of the 2017 ALCS.

So perhaps not a great comparision with the Rivera-era Yankees."

True, but Rivera had some rather famous blown saves in the playoffs as well. A time traveling Aroldis Chapman may well have struck out Luis Gonzalez in 2001 or David Ortiz 2004. Not saying he would have, but we also don't know that time traveling Mariano would have done better in recent years either.

or Sandy Alomar Jr. in 1997.

also, Count the Ringzzz is not a great look on the propaganda scale.

Chapman's no Billy Wagner, but he's allowed just two fewer runs in 27 playoff innings than Rivera did in 141. I think Mariano has him covered in the postseason, no matter how many Alomar offspring or Luis Gonzalii you can dig up.

BBREF season splits for AL 2005, of course!

Don’t some of the stat ideas you proposed already exist? I want to say BPro or FG has them. But the thing is...who cares? We already know that only one to three relievers could probably accumulate enough value to be HOM members. Basically you need to be Goose or Mo to get in. I doubt that Fingers would make it today because we have more granular metrics now, and I think Wilhelm might not either. He’d be real close. I wouldn’t vote for him.

Wilhelm has 2200+ innings with a 147 ERA+. That's likely in for me regardless of role. I just PHoM Roy Oswalt and he's roughly the same number of innings but 127 ERA+.

@Carl Goetz:

Rivera had some rather famous blown saves in the playoffs as well. A time traveling Aroldis Chapman may well have struck out Luis Gonzalez in 2001 or David Ortiz 2004. Not saying he would have, but we also don't know that time traveling Mariano would have done better in recent years either.

Maybe, sure. But that's a lot of 'what ifs' for my taste. The fact remains that Mo did deliver in a lot of high-pressure postseason appearances, and Chapman not so much. Or, as @SoSH U at work puts it: "Chapman's no Billy Wagner, but he's allowed just two fewer runs in 27 playoff innings than Rivera did in 141."


@Dr. Chaleeko:

Don’t some of the stat ideas you proposed already exist? I want to say BPro or FG has them.

Details please!

But the thing is...who cares?

Um... don't we?

We already know that only one to three relievers could probably accumulate enough value to be HOM members.

Sure... but... don't you want to know which ones? Isn't it interesting to find out if Fingers, say, was a mistake, or if Hofffman or Wagner meet the standards?

Isn't the Hall of Merit explicitly purposed to figuring this stuff out? Isn't it why we're here?

DL,

There’s more to Wilhelm than his IP and ERA+. As a knuckleballer, he had a lot of WP/PB which lead to UER runs, so RA9 is more apt. But also, even tho Wilhelm went longer in games, the benefits of relieving still apply to him.

Also, with respect to Wilhelm, he accrued a lot of value, but outside of his year starting (which maybe he should’ve done more of), he doesn’t accumulate a lot of value any particular season (IOW, a lack of peak). Not once do I have him as the best reliever in baseball in any season. By contrast, I believe I have Rivera at 5 such seasons.

I’d much rather have Gossage’s value spread over 10 peak-prime seasons that Hoyt’s spread over 20 (which explains why Goose barely makes my PHoM and Wilhelm doesn’t).

Just found this & wanted to share it, as it's certainly germane to the discussion here.

@Bill James:

The proposition that to win a World Championship you need a great closer, and that a great closer is more important than a great player at the other positions, appears to be true.

Analysis here: https://www.billjamesonline.com/the_all_important_closer/?AuthorId=3&pg=9

Analysis here: https://www.billjamesonline.com/the_all_important_closer/?AuthorId=3&pg=9

A reader I presume is Primate GuyM has some well-founded issues with Bill's study.

A reader I presume is Primate GuyM has some well-founded issues with Bill's study.

Under whick moniker please?

Under whick moniker please?

Guy123.

Right... Thanks very much for pointing out that interesting back-and-forth.

Save opportunities, on which Guy123 spends quite a lot of time, strike me as 100% irrelevant. As James points out in the thread, "Save Totals are a minor element in how Closers are evaluated." They do, I suppose, quickly point out which staff pitcher is the designated closer - but that's about all. I regard saves as a very poorly constructed stat devoid of analytical meaning. In 2007 Wes Littleton 'earned' a save by 'protecting' an eleven-run lead (which then ballooned to 27 runs) for three innings.

I can't say that I consider saves at all for analytical purposes. This was in fact a primary prompt for my starting this thread - because if saves have no analytic worth, then how, exactly, do we evaluate relievers? Personally, I think a combination of WHIP and some form of the stats I propose above will paint a far more accurate picture than saves (or holds, for that matter).

Guy123's second point - that relievers on top teams may be more likely to be top-rated by virtue of being on a top team, and that assigning them a top ranking may therefore constitute circular reasoning - strikes me as much more germane. But that same argument could, I strongly suspect, be constructed for other positions on the field as well. I still think James's central point stands.

Bill James reliever rankings are based on Saves.

Bill James reliever rankings are based on Saves.

No offense intended, but that's factually incorrrect.

The all-time saves leaders run Rivera - Hoffman - Smith - Rodriguez - Franco.
James's rankings run Rivera - Fingers - Hoffman - Gossage - Wagner.

@Bill James:

I don’t intend to explain the points, because that’s a four-page explanation.

Further:

The points are really just displayed here to show how absurdly far ahead of everyone else Mariano is.

Mariano is ahead of Hoffman by 8% in saves, but by 70% in James's ranking system. On the other hand, Hoffman is 76% ahead of Fingers in saves, but 5% behind Fingers in James's system.

This is actually a rate stat advantage for middle relievers relative to closers.

I don't understand your point at all. Since inherited runners scoring has no effect on a relievers rate stats, how does this create an advantage for MR?

Because middle relievers can allow inherited runners to score with no ERA penalty. Which, in terms of accurate evaluation of a relievers' effectiveness, is ridiculous. That's why I advocate what I've chosen to call RRA (Relief Run Average), which would tally both earned and inherited runners.

James' chart suggests - sample size issues aside - that you need a good bullpen to win a title. but he doesn't seem to realize that.

if even your best reliever - which thru the 1970s-2000s was virtually always your SV guy - is not that great, then by definition the rest of your bullpen is even worse (as Guy123 noted).

this effort is quite a reach anyway, but a more sensible approach would have been to examine how good the main 3-4 relievers are on the title teams.

comparing the quality of closers to everyday players is inherently silly. now, if even the best 2Bs or Cs or LFs were almost always lifted partway through games, or if the closer pitched multiple innings, that would be one thing.

if James wanted to compare something to batters who play 150 full games a year, then it is plainly obvious that the comparison would be "bullpen" and not "closer."

Bill is an all-time great, but he whiffed on this one.

James' chart suggests - sample size issues aside - that you need a good bullpen to win a title. but he doesn't seem to realize that.

if even your best reliever - which thru the 1970s-2000s was virtually always your SV guy - is not that great, then by definition the rest of your bullpen is even worse (as Guy123 noted).

this effort is quite a reach anyway, but a more sensible approach would have been to examine how good the main 3-4 relievers are on the title teams.

comparing the quality of closers to everyday players is inherently silly. now, if even the best 2Bs or Cs or LFs were almost always lifted partway through games, or if the closer pitched multiple innings, that would be one thing.

if James wanted to compare something to batters who play 150 full games a year, then it is plainly obvious that the comparison would be "bullpen" and not "closer."

Bill is an all-time great, but he whiffed on this one.

I think this starts to get at the issue with James's study. A World Series champ generally needs some good hitters. But you can get your good hitters at any of 8 positions. You need good starting pitchers, but here James does a weird thing of looking at left-handed starters and right-handed starters separately, which makes no damn sense to me. And, obviously, it helps to have a good bullpen. Now, I'm not entirely sure how he defines "closer" for his study, but if he defines closer as "best reliever", then, as Howie says, he's essentially discovered that a good bullpen is helpful to winning a World Series. If you have a lousy closer - especially if closer is defined as "best reliever" - then you can't have a good bullpen without having a good closer. But you can have good starting pitching without having any good left-handed starters. And you can have good hitting even if your second baseman doesn't hit worth a damn. So that, ultimately, I think the comparisons across positions are in-apt, which leads to James's finding.

I don't know if I explained that well. But I think the basic set-up of the study is flawed.

Part I—Decisions
10 times wins, plus 3 times saves, minus 5 times losses
Part II—Earned Runs
Earned Runs Saved as compared to a pitcher pitching the same number of innings with an ERA of 5.00
Part III—Strikeouts and Walks
2 times Strikeouts, Minus 3 times Walks, the total divided by 3.

154 pitchers have 100+ Saves through 2018, one more season than the linked rankings and I'm not going to filter out starting seasons, IP are slightly off because it's counting .1 and .2 as a tenth and a fifth:

90261 - 30087 Saves
43770 - 8658 Wins and 8562 Losses
26824 - 144372.7 IP and 53383 ER
24728 - 116457 K and 52910 BB
185583 points

Around half the points come from Saves, if you're in a setup role for part or all of your career or if you retire before almost all the 40 Save seasons have occurred those are huge penalties. The comment on Nathan points out the importance of high value seasons which further elevates modern closers relative to all other relievers. Cy/MVP is factored in and there are apparently 4 pages of calculations but it's "based" on Saves.

Cy/MVP is factored in and there are apparently 4 pages of calculations but it's "based" on Saves.

Forgive me, bbmack, but this is where we get into precision of language.

You make great points in your posts. I especially acknowledge this:

If you're in a setup role for part or all of your career or if you retire before almost all the 40 Save seasons have occurred those are huge penalties.

I consider this the most cogent argument against James's study I've read in this thread. This point probably persuades me against the validity of the study.

So I'm not knocking you. I just wanted to make that clear before I continue.

James's study gets 48.6% of its point value from saves. That's not "based on".

Pendantic? Of course it is. That's because this discussion is rooted in mathematics and science. Math and science are pedantic. Non-pedantic maths and science reduces discussions like these to he said vs. he said squabbling - which is, in large measure, what ended up happening to the 2019 ballot thread discussions on relievers.

"Based on" has a specific meaning that is not met by your statement. ERA is based on earned runs and innings pitched; James's study is not based on saves. "Based in part on" would be accurate; "based largely on", or "based primarily on", would be inaccurate, as the baseline is under 50%; "based on" is simply wrong. Any teacher (I'm a teacher) would mark your statement as wrong on a paper or test.

(Ironically, James himself, who I revere, is frequently guilty of this kind of hyperbole. It's the #1 thing I find frustrating in his writing. And if I'm comparing you to Bill James in any way whatsoever, you're doing OK in my book.)

This is a diversion from the debate, I know. But many of the posts on relievers in the 2019 ballot thread were intensely frustrating because they used hyperbole in place of precise language, and I really, really want to avoid that here. Please keep making your excellent points!- and please, don't hesistate to call me out when I stray from the same standard.

Any teacher (I'm a teacher) would mark your statement as wrong on a paper or test.

Everyone on this board is now dumber for having read it. He awards you no points, and may God have mercy on your soul.

Everyone on this board is now dumber for having read it. He awards you no points, and may God have mercy on your soul.

LOL! Fair enough.

I went with based on Saves because it provides the variance for almost all the relevant pitchers. Non-save point leaders:

3046 - John Smoltz 3473 IP career, 3211.2 IP in Starts
2525 - Dennis Eckersley 3285.2, 2478.1
1736 - Hoyt Wilhelm 2254.1, 383.1
1538 - Ron Reed 2477.1, 1603
1467 - Lindy McDaniel 2139.1, 445.1

1414 - Mariano Rivera 1283.2, 50
1374 - Goose Gossage 1809.1, 252.2
1320 - Rollie Fingers 1701.1, 195.2
1302 - Tom Gordon 2108 IP, 1260.2

No idea what ERA under 5, Wins, Losses, K and BB do for you. Lindy McDaniel has a lot of relief IP if you didn't already know. 10th is Stu Miller 1042 and 90% of that is 21st Jesse Orosco 940, 80% of Stu is 32nd Bruce Sutter 841, 70% is 47th Kenley Jansen 736. Since Stu isn't a modern closer overall he drops to 35th with 1501.

With Saves:

3695 - Dennis Eckersley 3285.2, 2478.1, 11th on ranking
3508 - John Smoltz 3473, 3211.2, not on ranking
3370 - Mariano Rivera 1283.2, 50, 1st

2744 - Trevor Hoffman 1089.1, 0, 3rd
2420 - Hoyt Wilhelm 2254.1, 383.1, 7th
2343 - Rollie Fingers 1701.1, 195.2, 2nd (Cy/MVP bonus specifically mentioned)
2314 - Lee Smith 1289.1, 37, 9th
2304 - Goose Gossage 1809.1, 252.2, 4th
2303 - Billy Wagner 903, 0, 5th

2184 - John Franco 1245.2, 0, 12th
2171 - Francisco Rodriguez 976, 0, 8th
2127 - Joe Nathan 923.1, 162.1, 6th
1989 - Lindy McDaniel 2139.1, 445.1, 23rd
1950 - Rick Aguilera 1291.1, 551, 45th
1905 - Jeff Reardon 1132.1, 0, 32nd
1894 - Jonathan Papelbon 725.2, 16, 10th

To find the reliever ranking Top 10 it takes going all the way down to 16th without even removing starts or 4 pages of calculations. The 4 pages do create separation if you want to trust the black box:

804 - Jonathan Papelbon 790 non-Save points, 368 Saves
536 - Jeff Reardon 804, 367
526 - Randy Myers 496, 347
404 - Troy Percival 493, 358
370 - Jeff Shaw 609, 203
Under 295 - Brian Fuentes 612, 204

Papelbon is really special for some reason(s) and Myers closes the gap on Reardon. Because it's Papelbon the obvious guess is his 2006 is really really good and Myers has a 53 Save season while Reardon tops out at 42. So yes, there are some easter eggs on the list, Papelbon is better than his stats suggest. Aguilera's relief pitching is 1521 points which would be 30ish place if I stripped out starting from all pitchers is someone lesser than his stats suggest. Lindy McDaniel 1913 in relief, he was pretty terrible as a starter and his relief pitching was also apparently flawed. Brian Fuentes is a direct descendant of The Man from the Train.

There is over 1000 points of variance from the Saves stat, Billy Wagner is over 100 Saves ahead of 18th place on the career Saves list when Bill makes his rankings, it's really hard for anyone to close that gap, 300 non-Save points is Papelbon to Percival or Percival to Mike Fetters.

When I put them into my system, Mariano is a slam dunk HoMer, Wilhelm is a borderline guy, and Goose is a ways off. No one else is even close. As I said in my ballot, even if you don't think relievers belong, Mo belongs.