Jeane Dixon’s Got Nothing on Me
Let’s assume for a moment that I ran a restricted access pay
web-site that provided projections and player and team comments for the upcoming
baseball season. Now let’s say the homepage of this site provided the following
information as an enticement to sign up for the service:
“If you had subscribed to our site for the 2001 season, you
would have had the advantage of getting valuable pre-season evaluations like
these:
“It's unclear what Sandy Alomar Jr. does to improve over Mark
Johnson.”
“Having Juan Gonzalez as your sleeper sounds strange.”
“If Shawn Wooten can catch at all, he'd deserve a real long
look from the Angels.”
“The Ichiro projection is a somewhat educated guess that the
system spit out. It seems real high but this is someone who was hitting for
consistent averages virtually unprecedented in Japan. Who knows, but I guess
I'd count him as an above average corner outfielder from jump street until further
results come in.“
“As it stands now, the Rangers offense is obviously pretty
loaded led by the two Rodriguezes. But I think it's reasonable to put question
marks next to the projections of Galarraga, Caminiti, Mateo and maybe Velarde.”
“If the Astros think they need a bat off the bench, don't
rule out a reappearance of Orlando Merced.”
“Kerry Robinson can run like hell and is exactly the kind
of player LaRussa occasionally takes a shying too. Might get some PT.“
“I'd like to see what Kevin Millar would do with 500 At Bats.”
“I consider Jimmy Rollins to be one of the better prospects
around.”
Now I’ll state that this site indeed had posted these comments
before the season and in fact have shown themselves to be more or less right
on the mark. Many of them could have some value to a roto-player, and others
can be seen as unexpectedly accurate (like the Kerry Robinson comment).
The picture the site would have painted for you would be very
pretty, would it not? Just sign up for the site, snag up all the little gems
of insight and cruise easily to the title in your roto-league.
Of course, life is never that easy. The above comments did
actually appear on a site before the 2001 season though the site offered them
up for free. The site which provided them was mine, and as such I can tell you
with some authority not to be overwhelmed by the apparent accuracy of the comments.
You see as the guy who wrote all those comments, I can tell
you that I wrote at least 100 more such comments, none of which were printed
above. Many were simple explanatory comments to use with the projections. A
good number of them turned out to be irrelevant, either the comments accuracy
never got a chance to be tested or there wasn't really anything of import said.
Of course some of them turned out to be “wrong”:
“A lot of people have written the Reds off and I think that's
a mistake.“
“If someone asks me, ‘Voros, who is your number one sleeper this year?’ My
money would be on Chris Donnels.”
“Michael Tucker is starting to show flashes of the player
many thought he'd become.“
Now if I was trying to sell you my site, you’d be sure not
to see those on there. And that is the problem with judging assessments by picking
out various individual comments from a group of many. Quite often you’re provided
with a limited view of everything that has been said.
Here’s a good example: I want all of my readers to pick a
number from 1 to 50 , with the only restriction being that both digits in the
number have to be odd. You have your number? Okay, here it goes…
…37…
How’d I do? Now many of you are sitting out there going, no
you idiot, you were way off.
But there’s also going to be a fair number of you whose reaction will be, “Wow!
Good guess! How’d you do that?” The key is that the people for whom I was correct
wouldn’t know anything about the people for whom I was wrong. As far as they
know I’m batting 1.000.
They are gambling tout companies who use this shady trick.
They have several “independent” touts front for them, knowing that at least
one of them will do very well over the season by chance alone. Then the next
year they advertise the incredible season that tout had last year, meanwhile
having all their other independent touts continue making picks. Then the next
year, the guy with the big win percentage becomes the expert. Every year the
company will have at least one tout that they can advertise as having tremendous
success last year and not actually be lying. I had a friend who fell for this
con one time and before making his bets he was convinced that he had tapped
into somebody others didn’t know about and who could produce remarkable results.
My friend didn’t lose a lot, but he lost enough to ignore anything the guy had
to say from that moment on.
Bill James wrote an article on ESPN.com recently on the subject
of people selectively picking out decisions that turned out badly for MLB GMs.
However, he didn’t talk about the kind of nefarious activity described above,
but rather of a mindset of a certain set of fans to dwell on the negative about
the teams they root for. Where I believe James erred was in putting himself
above this kind of behavior. This isn’t the behavior of a distinct individual
group of people. No this is really one of the most fundamental characteristics
of human behavior: the search for confirmatory evidence of one’s beliefs. All
of us (myself and Bill James included) engage in the behavior James describes
(ironically one could argue that James is engaging in it in the article itself).
There are a few theories in Psychology as to why this is, and several related
theories as to what times and instances this behavior exhibits itself most strongly
(all of which have nothing to do with baseball and I probably don’t understand
them well enough to describe them anyway). To be sure, this is not the behavior
of some deranged egomaniacal anti-fan.
Using myself as an example (as both someone who engages in
this behavior and also as someone accused of being a deranged egomaniac), I
know that I’m likely to rationalize away the disappointment of Chris Donnels’
season. The way I do it may contain perfectly valid reasoning (e.g. he didn’t
get many at bats and I did comment that the projection was a bit iffy), but
the fact that I don’t take the same pains to rationalize away my correct predictions
(e.g. predicting Juan Gonzalez would hit well isn’t hard and while Sandy Alomar
hasn’t been great, Mark Johnson might not have enough at bats so far for someone
to conclude he’s been better). It’s fairly standard behavior for someone to
scrutinize why their incorrect predictions aren’t as bad as they appear, but
rarely do we talk about how our correct predictions aren’t quite as good as
they appear.
The thing is, people shouldn’t be made to apologize for such
behavior, as James seems to suggest. It’s like apologizing for creating mucus,
or wasting time sleeping eight hours a day. We’re human, that’s the way we work
and there are good reasons for why that is. Fundamentally, that’s why concentrating
on “who said what” provides very little insight into finding the answers to
life’s (in our case baseball’s) questions. We are all subject to these kind
of flaws, so we are all going to have these moments where we fail to think or
speak logically about a situation. This is fine. The key should be to shift
the focus away from the person indulging in faulty logic and instead focus on
the faulty logic itself.
I’m sure Billy Beane has made decisions in the past that the
Drive-Thru guy at the Jack in the Box has disagreed with. I’m also fairly sure
that, at times, the Jack in the Box guy turned out to be right. The fact is
that the dumbest of us are often right and the smartest of us are often wrong,
and that the things we say and the points we make should be evaluated on their
own merits, rather than on some imaginary scorecard which keeps track of how
often we’ve been right in the past.
The advancement of everyone’s knowledge of baseball, be they
“stat-head,” “baseball insider,” “sportswriter,” “Joe-fan” or “talk-radio junkie,”
would be greatly increased if we stopped concentrating on who was a “stat-head,”
“baseball insider,” “sportswriter,” “Joe-fan” or “talk-radio junkie,” and started
concentrating on their respective arguments. Then again, what the hell do I
know? I’m just another arrogant stat-head.
Voros McCracken
Posted: August 27, 2001 at 01:00 AM |
3 comment(s)
Related News:
General
Reader Comments and Retorts
Go to end of page
Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.
An important side point, though, is that people aren't necessarily wrong in their perceptions when they do so. The people who think that the Royals are being badly managed are clearly correct, as their lousy W/L record clearly demonstrates. The wrongness is is focusing on single events as proof and/or confirmation of our own theories of where they're going wrong instead of stepping back and looking for long range patterns.
In many cases, the reason that people treat the situation that way is because they don't have a pre-existing basis for doing that kind of comparison. There's no standard groundwork for how to judge the behavior of a whole major league organization, and the problem is big enough that most people don't know where to start. Instead they get the general picture- and you don't have to look past the Royals' record for the past few years to get that picture- and start trying to find individual pieces of evidence that would help to explain the pattern. We know that the team isn't doing well, so we look for specific instances of things that they've done poorly as a first step of figuring out the general pattern.
If anything, the James article should serve as a caution that this approach is fraught with danger: the danger that we'll only confirm our pre-existing beliefs. Instead, we need to take a big step back and develop the tools needed to analyze any frachise, and only then apply them to the specific one in question. We should look at all of the general things that a team can do: signing amateurs, minor league development, promoting the right players, making trades, managing salaries, etc. Then we can figure out which strategies seem to be most effective, how effective they are, and so forth, and then turn around and see how the specific team in question is doing in those areas. By starting from the most general and working down to the specific, you minimize the risk of confirmation bias.
Meanwhile, Voros, your idea that the validity of the method, not the identity of the speaker, is what we should focus on makes sense in the abstract, but you can (and do) carry that too far. As humans, we have limited amounts of time. We need shortcuts, or our brains would explode from information overload. The identity of the speaker is one such shortcut. This doesn't mean we need accept everything Bill James says at face value, or that we need cover our ears when Harold Reynolds speaks. But you should pay more attention when a medical doctor diagnoses your ailments than when I do, and should pay more attention to my legal advice than to Dan Szymborski's. (If you pay me. Otherwise, I'm likely to deliberately give you bad advice.) I don't know everything about the law, and I do know some things about medicine -- but you don't have the time to run your life under the principle that the validity of the advice is completely independent of training and track record.
You must be Registered and Logged In to post comments.
<< Back to main