For those that don't want to wade through the entire posts.

OPS

CHONE: 93-69
ZiPS: 90-72
Oliver: 86-76
Marcel: 81-81
Fantistics: 79-83
PECOTA: 75-87
Sporting News: 73-89
Steamer: 70-92
2008: 43-119

ERA

ZiPS: 100-62
CAIRO: 95-67
CHONE: 91-71
Steamer: 90-72
PECOTA: 88-74
Oliver: 82-80
Marcel: 81-81
Fantistics: 80-82
Sporting News: 73-89

The pitching results are kinda interesting. ZIPS doesn't seem to have crushed the other systems in the component stats, but won hands down things in which hits allowed (WHIP) and defense (hit part of whip and ERA) are included.

Pecota's been getting murdered since Silver left. Did Silver take some secret not-suck algorithm with him? Did someone mess something up moving Pecota from spreadsheets? The whole "We use 75th percentile projections for pitchers because the mean projections looked like ####\" fiasco last month isn't exactly filling me with confidecne.

Yep, that's me. I was just reading through this to catch some of my typos and then I was going to send it to you.

Re: #1, I am interested in figuring out what ZiPS does beyond component stats. Seems like it must have an edge on the other systems there. It also manages to do well on each of BB/9, HR/9 and K/9 and seems to them combine them in the right way.

#2 I gather that Pecota wasn't fully automated or well documented.

It's not exactly new to see a complex system perform better when operated by the developer.

#2 I gather that Pecota wasn't fully automated or well documented.

Yeah - I don't know if it changed before he left, but I recall Nate Silver saying in an interview that it was more or less a bunch of delicately tied together spreadsheets and macros.

The phrase I remember is "rat's nest".

To do it well it has to be. I mean in some ways that's how messes like the financial crisis get started. A more or less effective statistical model leaves the hands of the guys who made it (and therefore know where all the chinks in the armor are) and into the hands of folks who put the numbers in, wait for it to spit the numbers out, and then move on.

Back in the day (1999-2001) I did pretty well at the projection game and most of the time it was because I had a good system, knew where the holes were and when a player exploited one of the holes I made an adjustment (EG, the system didn't handle a succession of four or five 180 at bat seasons well). So it wasn't a case of changing projections you thought were wrong, it was a case of changing projections for guys who had a profile the system didn't handle very well. Early on there was a player like Darrin Fletcher that would cause significant adjustments to the system, but after a while I felt like I had the general idea down fairly well.

And Dan it warms my heart to see you using the old '$H' designation. I have a ton of projection sheets lying around with a bunch of '$BB' '$HR' '$SO' '$H'. I don't know why I chose the dollar sign to indicate the I was using one of my funky rate stats, but it's nice to see it thrown out there from time to time. :)

It's not exactly new to see a complex system perform better when operated by the developer.

Heh...so in other words, he was training on his test data. Always THE big issue with signal modeling --- how to keep yourself from adjusting your algorithm so that you do a "better" job of fitting your test set. Just as bad as incorporating the test data into the training procedure. Would wager Nate had to make some changes to fit in his last year, and the input conditions are no longer true. I would speculate that unless BP plays with the algorithm, it will probably have some late career "renaissance" season when the planets and stars align and the input conditions match once again....

The fact that Dan hasn't felt a need to fiddle with ZiPS in a long time makes me happy.

By the way Dan, since we have your attention, when are you finally going to do a podcast like you said you were? You must have the free time since you've been on Perpetual Post's podcast talking about which coins you're least likely to put into your mouth and how real life doesn't have as good an experience points system as online role playing games.

You're only 31? I figured you were at least 40.

You're only 31? I figured you were at least 40.

For some reason, people on the internet always think I'm about 10 years older than I am. I preferred being confused for 30 better than actually being 30, though.

I just wish I actually sounded 30 - I have this nasally, highish, young-sounding voice. I should take up smoking or tuberculosis or something.

I have this nasally, highish, young-sounding voice.

Did the Ira Glass comparisons put you off doing more audio Transaction Oracles? Because 1. Ira Glass is awesome, so I don't see the problem and 2. I thought it was pretty well-done.

(Listens.)
I like your voice, Dan - it's a good fit for a podcast type format. (And, yes, a bit like Ira Glass.)

Heh...so in other words, he was training on his test data.

More likely it is something along the lines of Dan's Dopson error. The developer has a better sense of cause and effect with the underlying code. Anyone can see the infinite Dopsons and know something's wrong. But Dan can, far more easily than anyone, understand what caused it to happen. Anyone can tell what the macro does, but the developer knows best what it's designed to do.

Right now there could, in theory, be a dozen things wrong with PECOTA. Maybe it was 100 problems at one time, and they were able to fix 99 of them, but in doing that they caused another 11 problems. I'm sure Nate would have been able to fix 100 of them without creating more problems, simply because he knew it better than anyone.

And, yes, a bit like Ira Glass.

Maybe it's the slight Bal'mer accent.

No, I don't mind the comparisons to Ira Glass, but when I'm on the phone, I get called "Ma'am" a lot and once in a while, a telemarketers asks to speak to my parents!

To actually do something in audio, I can't really do it by myself. If it's just me talking, I have a tendency to ramble on. I'm an odd enough speaker that people that occasionally use the term Dan-Sequitur.

Maybe it's the slight Bal'mer accent.

Did you hear that? I try not to have any noticeable accent. I do have a problem with certain words - just still comes out "jest" (I had a bad one on the last PP bit) and the first syllable of horrible still rhymes with car.

It's in the cadences, really, but you do noticably have the same accent as Ira Glass, who's also from Baltimore, so . . . that must be what it is. That, or an enormous coincidence.