Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Wednesday, February 25, 2009

Fack Youk: Gargiulo: He Also Picked Mickey Rourke…

“He symbolizes everything that disgusts me. Obviousness. Unoriginal macho energy. Ladies man…”

I’m a little late to the party on this one, but Nate Silver’s projections of A-Rod’s home run totals just don’t look right.

There are three glaring problems to me.

  1. Silver predicts a perfect linear decline, meaning each total is lower than the preceding year. This, of course, is the result you are going to get when you take his “top 20 PETCOA-comparable players” like Silver did. If you take any 20 players and average their careers, the total number of home runs is almost always going to decline with age. Increase the sample size and it will always decline. The only problem is, over that ten year period it is extremely unlikely that any individual player is going to have a perfect linear slide. I will bet anyone reading $100 that this doesn’t happen to A-Rod. First one to take the bet in the comments is on, we can iron out the details later.

  2. A-Rod is going to be able to DH at some point. Common knowledge would suggest that playing regularly in the field puts wear and tear on a body, draining energy in individual games and effectively shortening careers. Many of his comparables didn’t have that luxury and were driven out of the league because they were no longer well-rounded players, not just because they could no longer be effective at the plate and hit home runs.

  3. The problem with being on pace to be the greatest home run hitter of all time is that you aren’t going to have too many people similar to you.

Repoz Posted: February 25, 2009 at 02:18 PM | 23 comment(s) Login to Bookmark
  Tags: projections, sabermetrics, yankees

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. 1k5v3L Posted: February 25, 2009 at 02:46 PM (#3085743)
Silver is obviously not a commie homo-loving son of a gun...
   2. J.C. Bradbury Posted: February 25, 2009 at 02:54 PM (#3085763)
Silver predicts a perfect linear decline, meaning each total is lower than the preceding year.


That's not what linear means, and the projection is not linear. In fact, it's not even declining at a constant non-linear rate. Any projection ought to predict his home run totals declining by year, with the understanding that random events will cause the actual outcomes to fluctuate around the decline, sometimes resulting in totals increasing.
   3. jwb Posted: February 25, 2009 at 03:05 PM (#3085784)
Aaron's career HR totals were helped tremendously by his move to Fulton County Launching Pad at age 32. Rodriguez will get no similar boost unless The New Yankee Stadium (that's what we're supposed to call it, right?) plays far differently than expected. Or he gets traded to Texas.
   4. The Essex Snead Posted: February 25, 2009 at 03:09 PM (#3085790)
The New Yankee Stadium (that's what we're supposed to call it, right?)

The Bronx Bombshelter?
The House That Ruthless Avarice Built?
John Sterling's Liquor Emporium?
   5. Fack Youk Posted: February 25, 2009 at 03:26 PM (#3085808)
That's not what linear means, and the projection is not linear.


The author here. JCB - Could you go a little bit further into why that's not linear, just for learning purposes?

Fair point about the projection consistently declining and not accounting for random fluctuations. But if you are always assuming a decline, doesn't that skew the projection downward? If it starts at 33, that is his highest total going forward? Seems unlikely.
   6. DCA Posted: February 25, 2009 at 03:34 PM (#3085812)
Linear means constant decrease each year, e.g. 21-17-13-9-5-1. The word you're probably looking for is monotonicity, which means constantly non-increasing or non-decreasing, but nothing about the rate of change.
   7. Rodder Posted: February 25, 2009 at 03:44 PM (#3085830)
3. The problem with being on pace to be the greatest home run hitter of all time is that you aren’t going to have too many people similar to you.

This is the most valid point to me. It's kind of like with Jamie Moyer - I don't pay any attention to his projection. How do you even project his numbers? All the pitchers his age I can think of are knuckleballers.

Note: After checking Moyer's b-ref page I see his nearest comp by age is Tommy John, who fell off the shelf for his age 46 season. Still, it is not much of a sample size.
   8. Fridas Boss Posted: February 25, 2009 at 04:13 PM (#3085861)
Fair point about the projection consistently declining and not accounting for random fluctuations. But if you are always assuming a decline, doesn't that skew the projection downward? If it starts at 33, that is his highest total going forward? Seems unlikely.

What's the alternative? Try to predict the up and down bumps along the trend line?

Projections aren't meant to get into that level of granularity. Doesn't stop the critics from using that as a means of outright dismissing them in favor of their infinite personal wisdom.
   9. jwb Posted: February 25, 2009 at 04:41 PM (#3085896)
Really, the only alternative you have to pick small enough sample size that an unusual single performance, say Hank Aaron's 40 HR at age 39, is enough of a perturbation to break the overall trend. I don't think this is desirable.
   10. Fack Youk Posted: February 25, 2009 at 05:39 PM (#3086013)
What's the alternative? Try to predict the up and down bumps along the trend line?

Obviously that would be foolish, and I'm not saying it should start at 55 either. But he published the results of the simulation on ESPN.com and opened up the discussion. It just one man's opinion, but I'm betting against it.
   11. Harry Balsagne Posted: February 25, 2009 at 05:50 PM (#3086029)
I remember watching an episode of Baseball Tonight (this was back when I was drinking DayQuil and Red Bull slings for breakfast, to numb the symptoms of my post-concussion syndrome suffered in the Octagon) and Eduardo Perez was saying that as A-Rod enters his peak, we will surely see him start to hit 80-90 HR per season. He was so sure of it, he actually said it twice. And no one else on the show batted an eye.
   12. Fridas Boss Posted: February 25, 2009 at 06:08 PM (#3086058)
It just one man's opinion, but I'm betting against it.

How brave. You get EVERY OTHER POSSIBLE COMBINATION OF HR TOTALS and he gets 1. Again, this shows you don't understand how projections work.
   13. Fack Youk Posted: February 25, 2009 at 06:15 PM (#3086073)
How brave.

Okay... I'll take the over?
   14. Fridas Boss Posted: February 25, 2009 at 06:30 PM (#3086106)
Over what? The total homeruns hot here forward? That doesn't address your complaints about the shape of the projects decline.

Here's what you need to do: Use whatever method you want and project EXACTLY what Silver does: how many more seasons A-Rod will play and the # of homers he will hit in each season.

Apples to apples, right? Then we can compare. Good luck.
   15. Walt Davis Posted: February 25, 2009 at 07:09 PM (#3086178)
This is the most valid point to me. It's kind of like with Jamie Moyer - I don't pay any attention to his projection. How do you even project his numbers? All the pitchers his age I can think of are knuckleballers.

That's not how projections are done. Projections are primarily based on a large multivariate model run on all players. ZiPS and PECOTA (and maybe others now) then do a small tweak based on comparable players which allows them to essentially run a small, localized regression.

The projection for someone like Moyer isn't based on a handful of knuckleballers, it's based on thousands of pitchers. It is a fair point to say that there are only a handful of "points of support" out at Moyer's end of the age curve and it's possible that the model shifts dramatically for post-45 pitchers ... but there's no reason to think it does and so Moyer's projection is basically an extrapolation of the standard model. Or to put it another way, while there have been only a handful of 46-year-old pitchers in baseball, there have been a lot of pitchers (and a lot of "old" pitchers) who've had 3 seasons fairly comparable to Moyer's last 3 seasons.

Hmmm ... maybe that could have been clearer. Let's just put it this way. This is oversimplified but essentially Moyer's projection looks like this (this should roughly be Marcel):

xFIP09 = .5 xFIP08 + .3 xFIP07 + .2 xFIP 06 + .25 Over32

So a weighted average of his last 3 years' performance and .25 penalty for being a year older and being "old". In the real systems, those coefficients would be based on a regression featuring thousands of pitchers. All that PECOTA or ZiPS does is maybe tweak that number a bit based on comps so if Moyer's comps didn't decline quite as much as .25, he'll get a bit of a boost but his projection will be somewhere between the above and the comps. (PECOTA and ZiPS underlying models are more complex than the basic Marcel above).

Or as Dan says in the new ZiPS intro, the comps are mainly just for fun.

Anyway, I can't access Silver's original article -- did he present the confidence bands? I'm guessing things start looking pretty silly pretty quickly.
   16. DCA Posted: February 25, 2009 at 07:24 PM (#3086208)
Walt - are you sure that's how the projections are done? Though I don't know, I like to imagine PECOTA is more like:

xFIP09 = (a) xFIP08 + (b) xFIP07 + (c) xFIP 06 + (d) age adjustment

where (a)-(d) are found from regressions over all players, but weighted by a similarity score to the player being projected, where the regression is unique to the individual and the coefficients could vary significantly from player to player (obviously, it depends on the scaling of the similarity score).
   17. DCA Posted: February 25, 2009 at 07:28 PM (#3086213)
It just one man's opinion, but I'm betting against it.

No, it's not. It's one man's "mean case" scenario. He's not saying that it's at all likely to actually come out exactly that way.
   18. Walt Davis Posted: February 25, 2009 at 07:32 PM (#3086219)
I will add though that I agree with the general sentiment that these projections aren't that useful for projecting individual performance. They're basically actuarial tables -- it's how the average player of a given age with roughly those last few seasons is expected to do. You see this in the distributions which ZiPS and PECOTA present. Dan's stopped doing the player spotlights but, for example, he did one for Lowrie in his Red Sox projection. He projects an 88 OPS+ for Lowrie but the 95% confidence interval around that projection is, roughly, 50 to 140. I coulda told you that with the added info that there's no way they'll play him if he puts up a 50 OPS+. Ellsbury's projection is basically the same. That's not really useful (which is not a knock against Dan). Even someone with a reasonable track record like Beckett has a 95% CI on his ERA+ of about 90 to 160. You don't need models to tell you that stuff though it should be comforting to Red Sox fans that the chances of Beckett really sucking are minimal (if only his injury risk was).

The standard error on the mean of a group will be quite small. So we can say with great confidence that players who've averaged a 120 OPS+ over the last 3 years will, on average, hit a lot better next year than the group who've averaged a 100 OPS+ over the last 3 years. But for any individual player, the projection is pretty much a crap shoot -- and that's one year out. Multi-year projections for an individual are kinda silly.

If memory serves, in the analysis of the Swisher-Betemit trade, although Swisher's mean projected OPS+ was about 15 points higher than Betemit's, Betemit still had something like a 40% chance of outhitting Swisher in raw terms (much less position-adjusted). Dan's confidence intervals aren't symmetric (good) but, roughly speaking, the SD on an OPS+ projection is about 15-20 points depending on how much data it's based on. The SD on the difference between any two players (assuming they're independent) is going to be about 20-30 points of OPS+.

We really shouldn't express as much confidence on player comparisons and projections as we do.

I'm reminded of some folks who were trying to sell some software for analyzing mortgages. They were very proud of their interest rate projection model. They boasted about how it captured the big downturn in interest rates in the early 00s. It was hilarious. All their model did was take the interest rate from 5 years before and project it forward to be the same with increasing variance. It was a random walk. 5 years out, they were projecting that interest rates would be somewhere between 2% and 16%. Well, duh, anybody in the room could have told them that. (And you wonder how the subprime mess came to pass.)
   19. Cuban X Senators Posted: February 25, 2009 at 09:16 PM (#3086377)
I will bet anyone reading $100 that this doesn’t happen to A-Rod. First one to take the bet in the comments is on,


Hmm, if I actually wanted to go to his site, I'd accept his wager of $100 and put forth mine of 46¢.
   20. Fack Youk Posted: February 25, 2009 at 09:45 PM (#3086415)
Here's what you need to do: Use whatever method you want and project EXACTLY what Silver does: how many more seasons A-Rod will play and the # of homers he will hit in each season.

Apples to apples, right? Then we can compare. Good luck.

Heeeeeey Fridas. Guess what?

http://fackyouk.blogspot.com/2009/02/dueling-projections.html

You got your wish!
   21. Rodder Posted: February 25, 2009 at 09:52 PM (#3086438)
Thank you for the information, Walt. You must have explained it well if even I understood it.
   22. Fridas Boss Posted: February 26, 2009 at 12:21 AM (#3086613)
It was certainly not my "wish". You were the one ripping a projection, it's good to see you're actually willing to put your money where your mouth is. Good luck.

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Dynasty League Baseball

Support BBTF

donate

Thanks to
Tuque
for his generous support.

You must be logged in to view your Bookmarks.

Hot Topics

NewsblogAlbert Pujols could keep playing to reach 700 career homers: 'If I’m close to it, why not?'
(143 - 2:28am, Mar 06)
Last: KJOK

NewsblogMLB suspends free agent Sam Dyson for entire 2021 season
(4 - 1:57am, Mar 06)
Last: Itchy Row

NewsblogSource: Former Boston Red Sox CF Jackie Bradley Jr. to sign 2-year, $24M deal with Milwaukee Brewers
(26 - 12:25am, Mar 06)
Last: Walt Davis

NewsblogBraves revenue fell by almost $300 million last year
(42 - 11:47pm, Mar 05)
Last: Joyful Calculus Instructor

NewsblogEmpty Stadium Sports Will Be Really Weird
(12233 - 11:13pm, Mar 05)
Last: Joyful Calculus Instructor

NewsblogPrimer Dugout (and link of the day) 3-5-2021
(9 - 7:47pm, Mar 05)
Last: puck

NewsblogUniversal DH and expanded postseason unlikely for 2021 MLB season, per report
(32 - 5:07pm, Mar 05)
Last: Joyful Calculus Instructor

NewsblogNBA 2020 Season kick-off thread
(1912 - 4:40pm, Mar 05)
Last: rr doesn't talk to pawns

NewsblogTrevor Story Rumors: Rockies Contract Extension Won't Happen 'Anytime Soon'
(17 - 12:38pm, Mar 05)
Last: Bourbon Samurai stays in the fight

NewsblogJoe Altobelli dies: Rochester's 'Mr. Baseball' led Orioles to last title
(14 - 9:49am, Mar 05)
Last: ERROR---Jolly Old St. Nick

NewsblogNew report alleges Terry Francona, Chris Antonetti knew about Mickey Callaway’s behavior
(50 - 9:12am, Mar 05)
Last: Jobu is silent on the changeup

NewsblogOT - Soccer Thread - Winter Is Here
(898 - 7:38am, Mar 05)
Last: OPS+

Hall of MeritMost Meritorious Player: 1936 Discussion
(20 - 11:07pm, Mar 04)
Last: DL from MN

NewsblogNew York Yankees manager Aaron Boone having surgery to get pacemaker
(13 - 8:27pm, Mar 04)
Last: snapper (history's 42nd greatest monster)

NewsblogFinding a Fit for Jackie Bradley Jr.
(15 - 6:56pm, Mar 04)
Last: Jay Seaver

Page rendered in 0.3247 seconds
48 querie(s) executed