I agree with Matt that you shouldn’t use anything else for teams and pitchers.  For hitters, you should do it the way James/Woolner do it, and assume a typical other 8 batters, and then apply BaseRuns (unfortunately, they use Runs Created).  One day, all sabermetricians will eventually be using BaseRuns.  Hopefully, all the current ones don’t have to die off before that happens.

Tango,
what’s going to be the difference btw Pujols’ 2004 RC, XR, EqR and BsR?

I don’t know, and frankly, I don’t really care.  The only reason to use an inferior model is if it’s easier to use and gives results that are almost the same.  That’s why Linear Weights is so nice.  (I consider EqR, XR and ERP as inferior derivatives of Linear Weights.)

The James explanation of his latest RC version is head-spinning.  If James were to simply adopt BsR, everything would fall into line rather quickly.

The only reason to use an inferior model is if it’s easier to use

Presently that is the case wrt RC - because it is already in common usage in sabermetric circles.

I don’t really care if expected runs matches actual runs.  AFAICT, that’s not dissimilar to Pythag wins versus actual wins.

Now, this isn’t to say BsR isn’t better or whathaveyou, it just isn’t as convenient (to me).

Oh, but I like LWts too.

Right, basic RC is fine for something quick and dirty.  But, all of JAmes’ convoluted RC versions should be thrown out.

I’m a Linear Weights man. Should I switch?

The reason I wrote that fanhome post was not to start a “my run formula is better than yours” debate, Chris. It was simply a response to a few people who DO like the formula but didn’t have a source for an explanations or versions, etc.

Feel free to use whatever you want.

As a general priciple, though, this country was not built on people sticking with older, established ideas just because they’re already there.

And as far as BsR not being as “convenient” as RC, that’s sort of a self-fulling prophesy. How is it ever supposed to *become* convenient if people with good saber reputations, such as Chris Dial, don’t use it because it isn’t already convenient enough?

Oh well, I suppose leaders lead and followers follow. Tango is a leader, for one.

As a general priciple, though, this country was not built on people sticking with older, established ideas just because they’re already there.

Data, please. :)

Using BsR is patriotic. Why does Chris Dial hate America?

this country was not built on people sticking with older, established ideas just because they’re already there.

So BaseRuns will not appeal to the United Empire Loyalists amongst us!

God Save the Queen!

fra: I’ll always be a LWTS man, for the convenience.  But, when it comes to pitchers, you should be a BsR man.

And as far as BsR not being as “convenient” as RC, that’s sort of a self-fulling prophesy. How is it ever supposed to *become* convenient if people with good saber reputations, such as Chris Dial, don’t use it because it isn’t already convenient enough?

Well, let me ask this: Why should someone like Chris switch from something that’s worked well enough for him? How likely is it, in the typical study, that Base Runs will give him a different answer - and one that is more likely to be correct - to the question he’s trying to answer than a simpler model? If you want Base Runs to become the method of choice for any person doing any study (not saying that you do), that’s the evidence that you have to provide. For most people, there’s no reason to use a complex model when a simpler one will give you an answer with which you feel comfortable and which is likely to be accurate.

—MWE

Not to speak for David, but for me, I prefer a simpler model (LWTS) if it gives me reasonable results.  The basic version of RC fits that bill too.  So, I’m ok here.

But, any other RC version should be destroyed.  They have the same complexity level of BsR, but are inferior.  Out with the old dog.

And LWTS knockoffs should be thrown out for the same reason, and that includes ERP, XR, and EqR.  Those are newer and inferior.

I recommend reading Patriot’s articleson this subject.

—-“For most people, there’s no reason to use a complex model when a simpler one will give you an answer with which you feel comfortable and which is likely to be accurate.”

Well, I don’t know why a saber expert would call BsR “complex”, but whatever.

These “dismissals” of BsR (or whatever you want to call them—I would not call them ‘constructive critism)—seem to focus not on the actual virtues or flaws of the method, but whether there is an extra step of calculation involved, and how much extra effort is therefore implied for the lazy sabermetrician. It boggles my mind. It’s not like those versions I posted in that fanhome thread are carried out to the nth decimal place, or that a complex “step-wise” linear regression was involved. On the contrary, BsR is pretty simple compared to lots of other current stuff.

So, bottom line, I could care less if some people think using BsR is a bit too much work. If so, then don’t use it. I mean, I got it, guys. I am much more interested in feedback from people who have something worthwhile to say (either positive or negative) about the method itself. When I first noticed that the thread had been linked to in BTF, I was looking forward to some worthwhile discussion—there are lots more readers here than on fanhome. So far, I am unimpressed…

I agree with David.  All the criticisms I’ve seen on BaseRuns (whether now or in the past) boil down to: (i) looks too complicated, (ii) not much gain, and (ii) too late to the party.

It may look complicated, but other than Linear Weights, it’s no more complicated than anything else.  And, it’s much less complicated than any of the RC tech versions.

The gain exists in the extreme cases.  If you don’t deal with those extreme cases (lots of HR, high run environment, etc), you have little to gain by switching… other than being right and prepared for those extreme cases.  BaseRuns works well across the board, something that no other run estimator can claim.  That’s mighty impressive.

And it’s not late to the party… all those other ones were invited way too early.  They should be thrown out like the drunken sailors they are.  They have no place in a sabermetricians arsenal.  That Palmer, James, Davenport, Woolner, et al have not adopted BsR just makes this a longer process than it should. 

I look forward to the day when the current B component of BsR becomes obsolete, and is replaced with something more accurate.  I’d guess David agrees with that as well.

Oh, and Eric Van at SOSH has his own Contextual Runs, which, IIRC, has a similar framework to BsR.  Perhaps someone can ask him to stop by, and he can post his construction.

I haven’t said anything because, as with a lot of these non-commercial formulas, I don’t see how to integrate BaseRuns with with what I am already achieving with Linear Weights.

For example, I need a really simple and true-to-live example of how to use it to work out an individual’s contribution. Say, for example, how much was Claudio Vargas really worth to the NatSpos in 2003-4? I’ve got him using a Linear Weights method employing FIP, as having cost about 3-3.5 wins.

And I don’t really understand why or how it can be used to make a comparison involving an individual batter. It seems I don’t want to use it for that, but do the BaseRuns for pitchers add up to something that makes a direct comparison with Linear Weights for batters possible the way Pitching and Batting Runs did?

And why is Extrapolated Runs a drunken sailor compared with Linear Weights?

Well, I don’t know why a saber expert would call BsR “complex”, but whatever.

It’s more complex than basic RC.

These “dismissals” of BsR (or whatever you want to call them—I would not call them ‘constructive critism)—seem to focus not on the actual virtues or flaws of the method, but whether there is an extra step of calculation involved, and how much extra effort is therefore implied for the lazy sabermetrician.

I’m not being critical of the method itself, and if you choose to read my comments as criticism, so be it. What I am trying to do is explain what you - as the author of the method - must do if you expect the method to be widely accepted by other analysts. You cannot come in with an attitude of “my model is based on the right analytical framework, therefore it’s better and you should use it”. You have to show the advantages of using Base Runs over other models, and why it gives more reliable results than other models. Provide the data that validates your model and demonstrates that it’s superior to other models. That’s your job, as an analyst.

This exchange between Chris and Tango at the top of the thread sums up the situation perfectly:

Dial: Tango, what’s going to be the difference btw Pujols’ 2004 RC, XR, EqR and BsR?

Tango: I don’t know, and frankly, I don’t really care. The only reason to use an inferior model is if it’s easier to use and gives results that are almost the same.

Dial’s suggestion would be the validation I’d be looking to see if I were being asked to use a different model - not just for Pujols, but for a wide range of players. To dismiss a request for data that would help validate the model with the assertion (and right now, that’s all it is, as far as I can see) that other models are inferior makes it highly unlikely that the model will ever be accepted for wide use. If that’s what you want, then it’s up to you to start the process, by taking the time and effort to validate the model. If you really don’t care whether or not other people ever use the model, that’s fine, too - just don’t criticize those of us who expect a little more than just an assertion that this run-scoring model is, almost by definition, better than all of the others.

That Palmer, James, Davenport, Woolner, et al have not adopted BsR just makes this a longer process than it should.

I wouldn’t accept it, and I can bet that the people mentioned won’t accept it either, without seeing supporting data that validates the model, and demonstrates that it gives better results in many contexts and results no worse than the other models in all contexts.

FWIW: For a linear relationship, forcing accuracy at the extremes - which as I understand it is one of the selling points of Base Runs, the idea that only Base Runs accurately represents that .000 OBP/.000 SLG produces zero runs - tends to come at the cost of being less accurate in the middle of the distribution, where most of the data points reside. I’ve done enough linear curve-fitting work to realize that even when you know that a relationship *should* go through the origin, forcing it to do so usually distorts the relationship enough so that you’re like to produce a more accurate picture of the relationship by accepting the error at the extreme ends of the distribution.

—MWE

tends to come at the cost of being less accurate in the middle of the distribution, where most of the data points reside

This is not true.  Patriot, in the linked articles, has shown the RMSE of BsR to be right in line with everything else.  Even Clay Davenport did the same thing. 

His article says: BaseRuns has the highest correlation of all the measures, including his own EqA.  The RMSE comes in at second, behind EqA.  He strangely says: Here you see a big reason why we use EqA: because its ability to estimate runs scored from team and league data is unsurpassed.   Big and unsurpassed?  As far as I can see, all he did was show how close EqA and BsR actually are.

I’d also say his statement here is probably wrong:
However, if you look at the 30 years from 1871 to 1900, the same XR and BsR equations are getting more than 60% worse—their RMSEs shoot up to 34.16 and 33.41, respectively. EqA, in contrast, “only” loses about 50%, scoring a 31.69. EqA is less sensitive to the conditions of the times than many of the other metrics which have been “tuned” to fit recent performances, so it’s especially good for historical performances.

I’d bet the way he “tuned” each equation was done wrong, at least for BsR.  The tuning should be done only on the B portion of the equation, but I’d bet he didn’t do that.

In any case, what’s the point really?  David presents a model that works over a large range of data points.  I’ve shown that RC and LWTS break down at specific points, leading to impossible results.  That, by itself, should be enough.  The only impossible results I’ve seen so far with BsR, is with the OBA range at .500 to .800, and the run value of the triple being wrong.  That’s as bad as it gets.  RC and LWTS have far more problems.  They get a pass because of Palmer and James.  As smart and groundbreaking as those guys are, and they are, that’s not good enough.

fra: the run values of XR are wrong.  They’ve been “fitted” based on limited team data.  The best way to do the linear method is through a change in RE process.  The fitting method attempts to correct for missing information by over (or under) weighting the known variables.  With my full LWTS version, there have far more variables, and therefore, more real.

Oh, and in the “Score Rate (by OPS class)” chart I have here, which is based on empirical data, we see that in games where the team OPS is under .300 or over 1.000, (static) LWTS is horrible.  When the team OPS is over 1.000, RC is horrible.  BaseRuns on the other hand is nearly perfect. 

The HR-happy games is where BaseRuns shines, as it should, since that’s where it’s model is based on.  LWTS and RC are both completely wrong in its treatment of the HR.  BsR on the other hand is practically perfect.

Again, there are no critics that point to any kind of flaws to BsR.  It’s based on a mathematical certainty, it’s rooted in actual baseball, the results conform to empirical results, and the process is no more complex than anything else (outside of LWTS and basic RC).  As the George Tenet says, this is a “slam dunk”.

From what I understand, the method of calculating Runs Created in Win Shares, is one of the more accurate measures because it partially takes context into account (for example HRs with Runners on base). 

In any case, I’m glad that Matt posted this.  I’ve been aware of Base Runs, but it’s been a while since I’ve seen the formula.

—-“What I am trying to do is explain what you - as the author of the method - must do if you expect the method to be widely accepted by other analysts. You cannot come in with an attitude of “my model is based on the right analytical framework, therefore it’s better and you should use it”. You have to show the advantages of using Base Runs over other models, and why it gives more reliable results than other models.”

Well, I only seem to adopt that attitude when I perceive unnatural resistance by people who are, AFAIK, usually open and curious enough to look at the issue for themselves—IOW, to not depend on my writing ability, my saber credentials as compared to those of James, etc., or my degree of initiative to evangelize about the topic. I freely admit that I lack in all of those things. I do what I can. IMO, it is more the responsibility of people who aspire to be in leadership positions in sabermetrics to look for themselves at all of the analytical stuff (of a certain obvious quality) that is out there, especially something as fundamental as a run estimator, and not to just sit there waiting to be convinced without putting out any intellectual effort. There is enough information, I believe, in my little articles, and especially in the more technically impressive work on the topic by Tangotiger and a few others such as Patriot, that anyone complaining about the lack of evidence is….probably not paying enough attention.

—-“It may look complicated, but other than Linear Weights, it’s no more complicated than anything else.”

Tango wrote that, but I’m not sure I agree that Lwts is so simple. The final formula *looks* simple, but what about the process that you go thru to generate that formula? Lwts values cannot be created out of thin air. If you want lwts values for the 1972 AL, you have to have PBP data and a sophisticated program to analyze it. Or, you can use BsR to yield a good approximation. Either way, lwts is not as simple as it may look.

—-”[Well, I don’t know why a saber expert would call BsR “complex”, but whatever.”

It’s more complex than basic RC.]

Not inevitably. James’ basic B factor is TB. So, as a result of his effort to keep it so simple, he weights a BB at only .24 runs, and a 1b at over .56 runs. That’s a bit *too* basic for me. For just a bit more effort (including BB in his B factor, and weighting extra bases different that the first TB on a hit in the B factor), James’ basic version would have been better—but then it would have been just as “complex” as the basic version of BsR.

Monty’s point is just more par for the course.  Again, David’s attitude is being called into question.  Who really cares what David, or me, or anyone, has to say?  Why not let the work speak for itself, and comment on the work?  I see it as a compliment to BaseRuns that all the criticism is about things not related to BaseRuns.

monty, in my first response on this thread, I said that I wasn’t interested in a big debate about which run formula is best,  I just wanted to provide info about BsR for interested parties. I did not ask to have it linked here on BTF, and I was/am not on a mission to convert anyone.

As far as answering Chris’ initial question, I didn’t because I assumed it was rhetorical. Anyone who follows run formulas at all would know that Pujols is going to have about the same total, whether you use BsR, RC, EQA, or whatever. That effort on Chris’ part was a strawmwn, because nobody ever said that BsR will yield noticeably different results for actual MLB hitters, and he knew (should have known) that.

——-“I just don’t think the question “Why is this new metric better than existing metrics?” should be answered with “Why do you stick with older, established ideas just because they’re already there?” It smacks of “Are you afraid of the truth?”-style fanaticism”

You are mixed up. The statement about established ideas was not a response to that question you just made up there. It was instead a response to the implied question of “even if BsR is just as good or better, why should I bother to switch?”. To that type of question, my answer was appropriate and in no way fanatical.

Just to be clear, I did not claim that BsR is “better”. Read the linked Fanhome article, please, and tell me where I said that. I did claim that, because BsR works well in all possible contexts and the other formulas don’t, that this implies that BsR does a better job of modeling the scoring process. People can judge for themselves whether that makes the forula “better” than others. And if you don’t believe that claim, the only way to be convinced is just to plug some numbers into these various formulas and see for yourself.

...the more technically impressive work on the topic by Tangotiger and a few others such as Patriot, that anyone complaining about the lack of evidence is….probably not paying enough attention

But I’m not complaining about the lack of evidence. I had a quick look at Patriot’s articles and did not find them easy to understand. And it brings me back to my questions above, none of which were answered except a query about why XtrapRuns was not necessarily useful.

As I understand it, the following things are true about BaseRuns:

1) It is more accurate than other formulas.
2) It should be used to evaluate pitchers.
3) It should not be used to evaluate individual batters.

Fine, I’ll take (1) on trust. I’d like to see a real-life example of (2)—although, I guess one could respond DTFM instead of RTFA. And I don’t understand why (3) is so.

Well, my article #3, which I linked, shows why it’s better to evaluate pitchers (who themselves are “teams”).  Across the board, BsR does a better job of modeling how runs are scored, on a game-level.

The reason to not use individual batters is because batters interact with other batters and not themselves.  This is why I said you should follow the James/Woolner process (though using BsR and not RC as the framework). 

All questions I’ve seen have been addressed in my 3-part articles, which is front and center at my site.  Yes, Patriot’s work is hard to follow, but that’s because he gets into the real nuts and bolts.

In a nutshell, all non-BsR equations work by accident.  There’s no way any of the other equations would work if 10% (or 20% or 50%) of the PAs end up as HR.  They have carefully fitted their model so that it only works within the narrow range that is MLB teams.  Great (and bad) pitchers are outside this range.  For the non-linear equations, it’s not so bad at the low end, because they all converge to 0 runs when the OBA is 0.  Softball leagues are outside the scope of the other equations… but not BaseRuns.  This is not to say that the other equations should work for softball leagues, but to ask “why don’t they?”.  A ball game is a ball game.  BsR is not even tuned for softball scores, and yet it works. 

The two reasons that it works is:
1 - only BsR properly handles the HR (min run value of 1, and it does not continually increase as the run environment increases)
2 - only BsR ensures that the number of runs scored does not exceed the number of baserunners

Don’t you think these two truths should be adhered to?

BsR adheres to these truths, and it has an error level at the MLB team level that is similar to the other measures, and it’s complexity is on par with those.

OK, I went and skimmed briefly the the tangotiger articles. (I’m at work.) I was struck by one thing—most of the work referenced 1974-1990 pbp. However, as we all know, a different run environment emerged c 1993. Would this have any impact on any equations?

fra: no, that’s the beauty of BaseRuns.  The equations will hardly change.  As an example, I generated the custom LWTS values for a broad range of run environments, but using the 1974-1990 PBP data (4.3 RPG).  Now, concentrate on the 5 RPG column.  This is what BaseRuns says the run values should be for this run environment.  Now, go to the empirical LWTS for 99-02, and go to the last line.

Start comparing.  BsR says a single is .488, while the empirical says .474.  BsR gives the double as .781, compared to empirical of .764.  The HR’s BsR is 1.406, while the empirical is 1.409.  The BsR BB is .327 vs empirical of .330.  .195 for the BsR SB, which is the same as the empirical.  The out is -.304 for BsR, with -.299 for empirical.

My guess as to the difference in the single and double is that the rates of taking the extra base on singles and doubles is smaller these days (slower players?  smaller parks?) making those events less valuable.

Ideally, we’d have a run modeller that also includes “taking the extra base”.  So, what happens is that this is captured in the single and double values.  SB, BB and HR are not affected by the “taking the extra base” variable, which is why BsR was able to nail those so perfectly.

If you wanted run values for Pedro, you’d use that custom chart.

You both make sense, but it’s sad that Monty’s point is true.  We had the BsR debate already when I presented those articles at Primer and at Fanhome.  The debate is that there is no debate.  It’s exactly what I said in my post #14.

Those custom LWTS charts is baseball.  It’s exactly what baseball is all about.  It clearly show how valuable sac bunts are in low run environments, and how risky a SB is in a high run environment.  It shows how the HR’s value doesn’t go up indefinitely.  It shows how costly outs are in higher run environments.  Baseball is not RC, LWTS, XR, EqA or anything else.  They are all wrong, and work only under the conditions they were constructed.  Every event depends on the context, and only BsR can give you anything close to reasonable.

Like I said, the debate is that there is no debate (which makes this a nonpopular position with the BTF debaters… we need to debate something!).

We should be past this point, and only talking about how to get a better “B” component, and how to make leaders like James et al see the light.

OK, I’m going to play around with BaseRuns and come back and report. I like the idea of not having to adjust numbers to suit run environments.

But how does it integrate with fielding? Can I get a Total Player Rating using BaseRuns?

—-“OK, I’m going to play around with BaseRuns and come back and report.”

Yes, that I what I said in one of my last posts. Ther only way someone will appreciate (or not appreciate, as the case may be) the difference is to “play around” with this stuff. And the reason I may seem “defensive” is that I never try to critisize a metric unless and until I have done just that. And I consider it a waste of time to try to spoonfeed people who seem to have big mouths, but haven’t done any real legwork.

I just read about a new version of runs created, in the 2005 R Shandler “Baseball Forecaster” annual by Neil Bonner. He tries to give a weight to each batted ball, not according to the outcome, but according to a subjective estimate of the speed of each type of batted ball, and adjusted to the batter’s rate of making contact, etc. It sounds good in theory, but when you wade thru it and get to the bottom line of what he is doing (using the +1 method), in the end he has yet another linear formula which weights a 1b at only .42 runs, and gives much more weight to K’s (as opposed to BIP outs) than the typical -.01 runs which has been found in several independent studies. Yet he claims that it is more accurate than RC. So what’s wrong? It turns out that the new formula is developed from only the last 10 years worth of data, and is tested against the basic RC version. Since that RC version is not tuned to that 10 year period, that almost ensures a favorable result for the new version.

Unless I had sat down and waded thru what they were doing (it’s a bit of work), I would just be talking out of my a$s to critisize it. But I walked the walk, instead of just talking the talk. I certainly could be wrong and have made an error, but at least I got off my butt with Bonner’s new RC. That is all I expect of people who take it upon themselves to diss Base Runs. The comments I have seen here, even from such top talents as Emeigh and Dial, do not leave me with a feeling that they have done their “due dilligence.

Mike Emeigh, in particular, has the ability to just blow my BsR bubble. So, go ahead. Either way, I will learn something.

That sort of crap is not what I, or Tango, tries to pass off with BsR. But readers will never realize that until they get off their duffs and

Since my articles were mentioned, I figured I’d chime in.  My article on Base Runs is not an attempt to explain the basic BsR philosophy or methodolgy—Tango and David have each done that, mutliple times, better then I could.  Basically, my article assumes that you are already familiar with BsR and then shows applications of BsR that you may or may not find useful(like the Theoretical Team methodology similar to what Woolner does with RC in MLV or James’ post-1998 RC formulas, and how to find the linear weight values for a given team or what have you based on BsR).

My “accuracy” article, which is also avaialable form the page Tango linked above, is the one that has the most relevance in this discussion.  Mike Emeigh asserted earlier that methods that correctly estimated at the end points lost accuracy in the middle.  I give the RMSE for ten different run estimators based on all teams 1961-2002 except 1981 and 1994.  It included versions that used S, D, T, HR, W, Batting Outs, SB, and CS only so that all the methods were using the same inputs.  Here are the results for a few of note:
Regression for those teams….22.56
XR….........................22.77
BsR…........................22.93
ERP…........................23.15
Batting Runs(Palmer)..........23.29
EQR…........................23.74
RC….........................25.44
BsR holds its own with any other estimator when it comes to accuracy on regular teams.  And when it comes to accuracy at extremes…as Tango has shown, BsR is not perfect by any means, but it runs rings around the other estimators.

I also echo Tango’s comment about the search for a better B factor.  On Fanhome we(David, Tango, myself, and others) have discussed various flaws in BsR and potential remedies for them.  I don’t believe for a second that any of us are loyal to BsR for any reason other than that it is the best method in our eyes currently avaialable.  But speaking for myself, it does get a little annoying that after publishing an accuracy study, after Tango’s series, after all of the various stuff David has written, that people still seem to demand a thorough defense of the method without (apparently) being familiar with the previous works or having challenges or corrections to make to it.

It should also be noted that my version of BsR was based on the 74-90 PBP data.  Patriot then ran accuracy tests on 61-73 and 91-0x data, and BsR still did very well.

XR does well because it was fitted based on that data.  In order to do a “Fair” test, you should construct the models based on the same empirical data (say 74-90), and then test the models with the out-of-sample data.  I’d bet that BsR trumps them all.

Thanks to Tango for reminding me of those tests.  They were posted on the old Primate Studies, but they unfortunately appear to be lost to history.  I still have the data for a followup study that was similar: I used the 74-90 data to establish a linear regression equation for runs and a linear regression to find the B coefficients for BsR.  Then I applied those formulas to 61-73 and 91-02.  I don’t advocate regression as the best way to find the B coefficients, but here are the results anyway:
61-73
Empirical….22.87
BsR….......23.16
ERP….......23.60
Regression…23.82
91-02
BsR….......23.17
Regression…23.25
ERP….......23.38
Empirical….24.17
“Empirical” was Tango’s empirical LWs based on RE for 1974-1990.

—-“I also echo Tango’s comment about the search for a better B factor.”

I have mixed feelings about that. On the one hand, in an intellectual sense, it would be great to have a super-duper B factor, using exponents or whatever.

But on the other hand, the reason I wrote the fanhome piece was because people seemed to want simplification/standardization. Those new version only extend to 1 decimal place, yet they should give good results, and remove some of the “too complex” objections.

The thread on fanhome seemed to have been prompted by a slighty earlier thread on fanhome that requested a straightforward version of the formula and clear examples of how it works, how one would use it properly.  The examples part—taking the willing but clumsy reader (like myself) thru some simple examples, step by step—remains missing.  It’s been asked for a few times in this Primer thread, and it remains missing.

(I hope Tango and MGL will be including clarifying examples of the various stuff in their book.  Generally, while I appreciate learning from each of them, Tango seems more prone to talking in ways that I don’t really know how to apply, while MGL tends to do more ‘let me slow this down for you’ clarifying in his posts—both the background reasoning and the practical use—and I’d be happy to get even more than that.)

Anyway, how ‘bout stepping thru a BsR computation of Dizzy Dean’s 1934?

On Fanhome, David says “If you want to tailor a version to a particular dataset (such as the 1975 AL), all you have to do is [tells general idea].”  But it would be very helpful, at least to this reader, if you stepped thru an example of doing that.  And computing Dizzy’s BsR number, and what to make of it, like compared to an average pitcher.

Just my $.02.

Since the run formulas in use are all in the same accuracy ballpark, use whichever one you like, and sleep well for your choice.

Please don’t kid yourself, however, that any one is inherently better because of the way it’s formulated. They all are oversimplifications of a complex matter. Each has its flaws and advantages.

As for XR, its greatest strength is its simplicity. You don’t need anything but the stuff in the boxscores and you’ll get good results. You don’t need the scoring context; you don’t need play-by-play; you don’t need the league totals; and you don’t need an advanced math degree. If it doesn’t suit your needs, though, use something else.

First, for SK, David defined the pitcher version of BsR as follows:
A = H + W - HR
B = (1.4*TBe - .6*H - 3*HR + .1*W)*1.1
C = 3*IP
Where TBe = 1.12*H + 4*HR

If we want to apply this to Dean in 1934, and you want to customize the B multiplier, first find the A, B, C, and D components for the 1934 NL.  So:
A = 11996 + 3247 - 656 = 14587
TBe = 1.12*11996 + 4*656 = 16059.52
B = 1.4*16059.52 - .6*11996 - 3*656 + .1*3247 = 13642.428
C = 3*10803 = 32409
D = 656
Calculate (League Runs - D)*C/(A - R + D) = (5695-656)*32409/(14587-5695+656) = 17104 This value is the B value necessary to make BsR = R for the 34 NL.  Take 17104/13642.428 = 1.254 This is your B multiplier.

So Dizzy has:
A = 288 + 75 - 14 = 349
TBe = 1.12*288 + 4*14 = 378.56
B = 1.254*(1.4*378.56 - .6*288 - 3*14 + .1*75) = 404.65
C = 3*311 = 933
D = 14
So his BsR allowed is A*B/(B+C)+D, or 349*404.65/(404.65+933)+14 = 119.6
He actually allowed 110 runs

You ask “what to make of it”, but that’s up to you.  You could calculate a runs/9 innings rate based on it, plug that into Pythagorean, whatever.  There’s no “BsR approved” method that you have to follow.  The BsR estimate is an estimate of how many runs the pitcher will allow.  Make of it what you will.

As for Mr. Furtado’s post, it is absolutely true that run estimators are all simplifications and that they all have flaws.  However, some are more flawed then others.  IIRC, you’re article on XR in the BBBA mentioned how you were dissatisfied with RC and ran through some of the problems with RC.

To take the fact that they all have flaws, though, and use that to say that none of the constructs are superior, is a gigantic leap of logic.  Base Runs does not have a perfect construction, but I think that a simple examination of the evidence would demonstrate to the satisfaction of most sabermetricians that it has a better construction then RC.

XR, like all linear formulas, are certainly very easy to calculate.  They have as good of accuracy with normal teams as other methods, and unlike RC or BsR they can be directly applied to an individual batter.  But if you try to take XR out of the range of normal major league data, it is not going to work.  If you had custom LWTS for every context, this would work.  And that’s what BsR attempts to give you—a formula that is accurate across a wider range of contexts.  Does it work perfectly?  Of course not.  But it does do a better job of adapting to different contexts then does XR or RC or ERP.

You also list a number of advantages of XR.  I have no quibble with these, other then to point out that most of them are true for BsR and other methods.  You don’t need anything other than the basic stats for a BsR formula; you don’t need to know the scoring context, or the play by play, or the league totals, or an advanced math degree.  You can use some of those things…you can set up BsR using all of the categories, just as there is an XR that uses all of the categories.  You can customize the B formula so that BsR precisely equals runs for whatever league you are looking at, but you don’t have to.

Patriot, excellent post.

To take the fact that they all have flaws, though, and use that to say that none of the constructs are superior, is a gigantic leap of logic.

No it’s not.

First of all, I’m going to try to be careful about what I write. I regret the tone of the criticisms of the other run formulas in my BBBA articles. In particular, my thoughts on RC and EQA should have been more restrained. When people expend a tremendous amount of time and energy and *do excellent work*, it’s important to keep things in perspective. Ultimately, my critisisms were quibbles that should have been framed accordingly. 

When you are talking about the run formulas at the top of accuracy charts, there really isn’t much difference between them. Sure, one might be a *little* better for a particular use, but none of them is superior for all questions. EACH HAS ITS FLAWS. XR has flaws; RC has flaws; EQA has flaws; LWT has flaws; and BsR has flaws.

So what’s an analyst to do? Should he expend hours and hours (or weeks or months) learning the flaws and limitations so he can gain a few runs of accuracy when he wants to answer a particular question? I think not. I think he can choose any of the best formulas and be confident he’ll generate excellent results.

XR, like all linear formulas, are certainly very easy to calculate. They have as good of accuracy with normal teams as other methods, and unlike RC or BsR they can be directly applied to an individual batter. But if you try to take XR out of the range of normal major league data, it is not going to work. (Emphasis added.)

In looking over my previous answer, I should have said “limitations” instead of “flaws.” Limitations better describes the differences.

If you want to know how many runs a team of Manny Alexanders will score, a linear formula isn’t your best choice. For the *normal* range of team and individual play, a linear formula works just fine. Since most other questions fall within the normal range, and since linear formulas are very easy to calculate and understand, I prefer them. Although I’m certainly biased, I prefer my XR because I known it gives good results whether I’m examining a low-scoring period (like the middle to late sixties) or a high-scoring period (like baseball in the ‘90s). If I want to compare players from the 1890s, though, I’d either have to do more research and create a specialized formula or find something else that does the job.

If you like something else, though, knock yourself out. But, do yourself a favor, understand the flaws of your prefered method. This way you won’t end up making claims your methodology can’t back up.

I’m a little hesitant about responding, because I think we are in a lot of agreement and it will seem as if I am nitpicking you by responding to only small parts of your post.  But that’s because I generally agree with the stuff I’m not responding to.

Anyway, you correctly assert that all run estimators have flaws.  Amen.  That still does not mean that they are all equally flawed or that none can be considered better then another.  Base Runs has superior accuracy to Runs Created with normal teams.  Base Runs gives seemingly more accurate estimates for extreme conditions than Runs Created.  Neither should be applied directly to individual hitters, but if you do, Base Runs will give you a result much closer to the more appropriate linear formula because it does not have the HR flaw that RC has.  I know of no formal situation in which using RC will do a better job of giving an estimate of runs scored then BsR. 

You mentioned that you regret the tone of your criticisms in your previous articles.  That’s your prerogative to do so, but in referring to your past articles, I was not questioning your tone, only pointing out that you had critically analyzed other methods.  I think that if sabermetrics is going to move forward people have to feel free to criticize and question methodology, as long as they remain respectful of the people who propose the methodology.  I don’t think that pointing out that RC is not optimal is in anyway an attack on Bill James or questioning XR is an attack on you or questioning BsR is an attack on David Smyth.

“If you like something else, though, knock yourself out. But, do yourself a favor, understand the flaws of your prefered method. This way you won’t end up making claims your methodology can’t back up.”

I agree with this, and it is my personal belief that Mr. Smyth and Tango and other BsR proponents have followed your advice.

I may have missed something in Sabermetrics 101, but is there a reason why the TB estimate for pitchers double counts HRs?  Wouldn’t it make more sense to use a multiplier for (H-HR) and then add 4xHR?

To use an extreme example, obviously a pitcher who gives up 10 hits, all HRs, will have given up 40TB, but his TBe would be 51.2.  Since base runs goes to the trouble of making sure that it counts the run value of a HR accurately, it seems odd that it could mismeasure the TB value of a HR.

BWC, good catch.  I’d bet that it is (H-HR).

If you only look at H-HR, and TB-HR, that ratio would come out to about 1.25.  (Not sure where that 1.12 is coming from.)

Looking at the equation that Patriot listed, and it doesn’t look right.

Oh, just thinking about it now.

If you have say 100 bases on a 100 singles, and 60 bases on 30 doubles and 15 bases on 5 triples, and you have 20 HR, you are trying to estimate the # of based on doubles and triples.

So, I’m right that about 1.25 * (H-HR) will do the trick (1.30 in my simple illustration).  However, 2b+3b are probably decently correlated to the # of HR.

Say then that that the rate of 2*HR / H = rate of (2b+3b)/H.  Now, the number of TBe becomes H + 4*HR + 2*HR*(1.2) ... (the 1.2 being the number of extra bases per double+tripl)

So, you have one equation that says:
(H-HR)*1.25 + 4*HR which becomes
H*1.25 + 2.75*HR
and another that says
(H-HR)*1 + 6.5*HR which becomes
H + 5.5*HR

Take the average of these two, and you get:
H*1.125 + HR*4.125

Thanks, Tango.

I wondered if it had to do with HR correlating to 2B & 3B.  Obviously, if you have actual TB data you’d prefer to use that.

Obviously.  Always use the information you have, and infer what you don’t by making reasonable estimates.

So his BsR allowed is A*B/(B+C)+D, or 349*404.65/(404.65+933)+14 = 119.6
He actually allowed 110 runs.

You ask “what to make of it”, but that’s up to you. You could calculate a runs/9 innings rate based on it, plug that into Pythagorean, whatever. There’s no “BsR approved” method that you have to follow. The BsR estimate is an estimate of how many runs the pitcher will allow. Make of it what you will.

“Make of it what you will” is part of the problem.  That seems to be saying that runs models produce estimates just for the sake of making estimates.

Just some random thoughts:
1. Is there a MSE for pitcher-level estimates?
2. If BsR is better because it’s accurate for MLB as well as Little League and church league softball games, then technically that makes it a better overall model of run estimation.  But it doesn’t follow from that that it should be the only model used at the MLB level.  Its estimates in the MLB range are merely comparable (and not superior) to ones produced by many other runs models.  So, unless it has some superior explanatory power at the MLB level, the use of BsR instead of RC, XR, or LWTS should be just a matter of preference.
3. I can apply BsR to Dizzy Dean, but what I want to know is why I should apply it to him or any other pitcher?  What does the estimate of 120 runs tell me about his 1934 season that I didn’t know before calculating it?  And, why would I want to plug the estimate into a runs/9 innings rate or a pythag when I know the actual number of runs he gave up?
Unless it’s purely a predictive model, a model should be able to tell users something about how the empirical world operates.  What does BsR teach about run scoring, and can I apply what I’ve learned?