Stacking the Deck

Bill and David take an in-depth look at the success rate of first-round draft picks.

“A manager has his cards dealt him and he must play them”—Miller Huggins

In holding the purse strings of Major League Baseball franchises, today’s owners and general managers face tough choices every day.  Decisions about player personnel alone require massive research, judgment, and as always, some good luck.  One age-old question centers around “nature versus nurture”—should teams focus their capital in human resources on scouting top talent of prospects from the high school and college ranks or focus on the development of bargain players through coaching?  Are the Mickey Mantle’s of the game born or bred?

We’ll look at today’s major-league baseball leaders in a variety of categories, tracing their origins to examine correlations in their picking order and current success.  Other issues might surface from the data too, such as, All-star potential and which team will likely win the World Series.  Surely, we won’t answer all these questions once and for all, but we intend to examine how statistical research of leading prospects can lead teams to allocate their resources more wisely, field better players, and hopefully win more pennants.

Clark Griffith, owner of the hapless Washington Senators, once said, “Fans like home runs and we have assembled the pitching staff to please our fans.”

Our overall hypothesis states that:

 

1. MLB Scouting does identify promising major-league players, as evidenced by the success of players who were selected during the first-round of the major-league baseball entry draft.  Moreover, predictability varies:

 

1. by "picking order” of top selections;
2. by position; and lastly,
3. by experience levels of selections, such as high school versus college.

“Baseball statistics are like bikinis—they both reveal a lot, but not everything”—Toby Harrah

Our population is all major-league baseball first-round draft picks, as published on the major-league baseball web site.  Our sample was collected from this list and includes all first-round draft selections from 1982-1986.  One advantage of this data set is that it is the complete population of the 130 first-round draft picks of 1982-1986.  The sample inferences will be made on career results of the population of first-round selections of the period, 1982-1986.  We chose this period as it is the most recent year for which the full term of its players? careers can be accurately assessed.  Indeed, the careers of most of these players are complete, and we can make fairly accurate assessments of the success of those that aren’t (Roger Clemens, Barry Bonds, Matt Williams, etc.).

“Scouting biases” are sure to exist to some degree.  Scouting biases relate to both the imperfections in the process of observing each new couple ballplayers, and also the passing trends or idiosyncratic priorities that shape a scout?s scoring system.  We believe the limitations and biases are inherent in the process, but importantly, that they are normally distributed for all players, inside or out of our sample.

“Baseball is 90% mental.  The other half is physical.” —Yogi Berra

The first part of the analysis will determine correlation of success with selection order, position and experience of our sample.  Before we pulled the data, we created a rating system of buckets to be detailed later in this study.  First, the sample is divided and grouped by defensive position at the time of selection— All Pitchers and All Hitters, classified as Catchers, Infielders and Outfielders.  The performances of the players are then compared against their peers, the population of MLB players of the same positions and career span.  The Sabermetric Baseball Encyclopedia software enables us to compared each sample player’s output to those of his peers at the same position.  Using “Plate Appearances” and “Innings Pitched” as our basis for hitters and pitchers respectively, we determined expected values for each statistic and compared to the player’s actual values.  For example, if in 1988, first baseman averaged one home run every 19 plate appearances, we would then apply that ratio to Mark McGwire’s output for the same year, and see that his expected value was 35 home runs, versus his actual output of 70 home runs.  This effort was then replicated for each year of each player’s career to determine what his career stats would have been had he performed equal to the average of his peers, versus what the player actually did.  Criteria and performance measures are chosen by us and are based on what we think are the most representative measures of success of the most widely used quantitative stats for hitters and pitchers.  Performance is then scored for each criterion on the basis of several factors—

• 10% or better than peers, called a Ten.
• 20% or better than peers, called a Twenty.

(Note:  Fielding performance, while important, is not considered because of its qualitative nature.)

Next, we assigned seven final achievement “buckets” to the output, based on longevity minimums and a matrix of the total “mentions” from above.  Ultimately, it is these categories that determine a player’s value.

We picked two increments of 10% above the mean as our significance level.  These particular levels needn’t be used, and will severely impact the hypothesis test; however, we believe these increments the fair and persuasive levels of significance.  Baseball fan and political type George Will argued in his 1990 book, Men at Work: the Craft of Baseball, that the big leagues have evolved uniformly to be more competitive and more precise than ever before.  If true, margins of greatness should diminish, and we’d allow the significance level of the test hypothesis to shrink over time.

The buckets are also our creations.  The names are descriptors only, for hitters and pitchers; the choice of seven buckets was our bet for the most efficient career breakdowns; the aggregate seasons are taken from batting title minimums; and the requirements are our attempt to extrapolate the qualitative judgment of a player’s career from the quantitative summation of his performance.

“Say you were standing with one foot in the oven and one foot in an ice bucket.  According to the percentage people, you should be perfectly comfortable.” —Manager Bobby Bragan

The sample mean for all first-round players is 3.02.

Results by picking order:

Overall, the higher the pick, the stronger the performance.  We bracketed picks of six and seven players in order to yield large samples of 30 or 35 players per group, over the five-year sample period.  We compared the means above, using large sample confidence intervals (z-scores), in each case tested at the 95% level.  According to our analysis, only the top grouping of first rounders has greater success than their peers, an indication in this case that their relative success is significant versus the lower picks.  Both subsets of hitters and pitchers also show the highest picks generating the strongest careers.  In addition, pitchers 14-20 were better than pitchers 21-26, as were hitters 7-13 versus hitters 14-20.

Results by position:

The data are less conclusive when sorted by position, but first, these position classifications deserve some background:  Naturally, catchers are grouped together easily enough.  All pitchers are lumped together at the draft level, as are all outfielders.  Second and third baseman from either high school or college rarely enter the draft; instead, each draft contains a glut of shortstops (20 of 31 infielders including first baseman are listed as shortstops), many of whom switch positions as managers and GM’s adjust the oversupply of able shortstops.  Examples of first rounders drafted as shortstops that are currently stars at other positions include Gary Sheffield, Chipper Jones, and Matt Williams.  Finally, a small percentage (less than 3%) of players are drafted across multiple categories (e.g. pitchers/outfielder, infielder/outfielder).  These are classified as “others”.  Using t-scored confidence intervals, the data from the table above shows only infielders have a significantly better chance of future success than prospects at other positions.

Results by experience:

For all players, college bred first rounders significantly outperform players coming out of high school.

For the subset of All hitters, college bred players also outperform their high school counterparts.

However, for pitchers, the results are “too close to call”.

Looking at combinations of position and experience, we see that high school outfielders as a group significantly underperformed the other hitting categories, except catchers.  One limitation of our data was the need to group all catchers together given their small total representation.  Catchers in both experience categories, seven from high school and six from college, produces too unstable a confidence interval for meaningful results.  Judging all catchers together, we see that they underperformed college infielders, college outfielders, and high school infielders.

Up to this point, the data shows that:

 

1. Picks from the top bracket of the first six players chosen outperform lower picks.
2. Catchers and high school outfielders have significantly weak track records in the majors.
3. College players consistently achieve greater success than high school players.

The second part of our analysis determines the representation of first rounders among baseball’s brightest luminaries over the decades.  To further evaluate the success of first-round draft picks, we took a look at the leaders in key statistical categories from 1981-2000 as shown above.

Given that first rounders comprise less than 2% of the pool of players entering baseball, the alternative hypothesis suggests that first round compositions would be at or near 2% (Central Limit Theorem).  Rather, the primary hitting categories all show levels at or higher than 28%.  Secondary and “other” hitting categories feature similarly disproportionate compositions of first rounders.  The corresponding pitching categories are less weighty on the first rounders, yet all still boast compositions at or better than 12%.  As stated earlier, first rounders comprise approximately 2% of all professional players.  If each player were equal across all draft picks, than expected value of 1st rounders in each top 25 list compiled would be .5 (25*.02).  In fact, in every table we see a significantly higher number.  We then applied binomial probabilities in each category to determine the likelihood of the actual value.  In every case, the likelihood was .000, or less than one in 1000.  To hone in on the data, we replaced 2% with 10% and 20%, assuming than that all players were equal over the 1st 10 and 5 rounds, respectively.  We still had very low likelihoods and random selection, less than 1%.  As a result, we conclude that 1st rounders are, as a class, more talented than their counterparts.  As seen in the results of the binomial distribution, the odds of this success ratio in a random test are less than one in 1000.  We conclude from this data that a strong correlation exists between top performance and early selection.

The 20-year results from these time-honored baseball categories are consistent with the five-year player data previously discussed: first-round hitters and pitchers are more likely to distinguish themselves successfully at the game’s highest level, and that of the two player types, hitters provide bigger impact.

The three highest major-league baseball player annual awards are listed above.  The MVP, Rookie of the Year, and Cy Young Awards honor the top player, rookie, and pitcher, respectively, in each of the National and American Leagues.  Again, first-round players make up a significant percentage of each of the awards of the period.

The World Series and League Championship Series awards recipients are simply the series MVPs in the playoffs.  Since many managers will argue that “anything can happen in a short series”, and since the MVP is normally chosen from the winning team, one might expect that first rounders would not make impact here.  The impact is in fact less, but it is still felt.

Players are selected to the MLB All-star game based on a combination of three factors: their popularity among the fans (8 starting players per league as voted on by the fans—16 players per season); the relative success during that specific, and to some extent previous, season(s) (approximately 30 to 35 players per season); and their relative success to the players on their own teams (representing teams that have only their one mandated All-star, who may or may not have been on the team if the mandate did not exist—approximately 10 to 15 players per year).  The choices made by fans and managers represent crowning success on the field, and for many players, off the field to—as popularity can play as big a part in the fans? selections as achievement does.  To underscore the perceived value (in terms of wins and tickets sold) the teams receive from their all-stars, it is common for teams and corporate sponsors to reward players with extra bonus incentives for All-star selection.  The results of the last 20 years of data indicate a strong correlation between top picks and All-star selection.  Moreover, picking order within the first round strongly affects a player’s statistical chances of being named to All-star team: 42% of the first rounder all-stars were one of the 1st 6 selections of their year.

We fittingly end our look at today’s leaders with the winners of the Fall classic.  From 1981 through 2002, of the 18 World Series squads with the higher number of first-round picks listed on their season rosters (there are many ties), 12 have won that Series, or 66.7% of the time.  Since 1988, the numbers are even more remarkable—the team competing in the series with more first rounders on their roster won 75% of the championships, including the Diamondbacks over the Yankees and the Angels over the Giants the last two years.  If only through empirical observation, the high number of individual awards and All-star selections, along with the consistent characteristics of championship winners, all reflect the null hypothesis.

“Baseball is like church.  Many attend, but few understand.” —George Will

The scouting process does identify big-league talent at young ages.  Therefore, it is a wise focus for the front offices of baseball.  As a group, first-round selections in the major-league baseball draft have made a tremendous impact on the sport in the last 20 plus years.  Also, within the first-round, GM’s can look even closer to maximize their benefit and find the true nuggets, as defined by these basic principles…

 

1. When choosing between an infielder and any other position player, take the infielder.
2. Avoid catchers and high school outfielders.
3. When choosing between college and high school hitters, take the college player.

“Baseball fans are junkies, and their heroine is statistics”—Robert S. Weider been “In Praise of the Second Season (1981)”

The study has been informative for us in several ways.  First, the test of means and confidence intervals we created confirmed the null hypothesis more persuasively than we had anticipated. 

When comparing today’s leaders to our sample, we had to use vague, “conservative” percentages of first rounders.  Had the results margin been narrower, our estimates would require greater accuracy.  Other lessons include the growing biases, errors, assumptions and limitations that affected our test.  At first, we believed the endless statistical research widely available on professional baseball would help us resolve our issue relatively easily, but we uncovered quite a lot of details to address—absence of accepted standards of excellence, a lack of consistency in the raw percentage of first rounders in the majors, are examples of details that require time, and threatened to alter the results.  Choosing a sample that would remain large even as we narrowed our focus would have provided more conclusive results.  Our initial sample of 130 turned into sample sizes in the teens as we looked at combinations of our factors—experience, position, and pick number.  The insights and lessons together underscore for us the value of statistical research—“there’s always more to be done, and the results may prove surprising.”

From here, one might want to reconstruct the analysis and further ways:

Naturally, the process can be duplicated for lower rounds, or grouping rounds together.  We resolved only that the scouting process is a wise allocation of a team?s operating budget, and further analysis on the return on investment of players in different draft levels can be useful in determining optimal spending levels.

In addition to test of means, we recommend regression analysis to:

 

1. Determine significance levels of the buckets for the dependent variables we chose—picking order, position, and experience.
2. Judge the significance of qualitative variables (injury, popularity, leadership, etc.) as a whole, on the draft selections.
3. Re-create the analysis without one of the variables to determine the highest correlation values of the variables used, and adjust model accordingly.

It is now up to the GM’s determine how best to use this data.  However, they should choose wisely, as if they don’t they must remember…

“There’s no crying in baseball”—Tom Hanks in “A League of Their Own”

This analysis showed us how many truly great players came from the club of first-round picks.  The thought of an All-star team of first rounders is irresistible to us, and therefore, we conducted a small survey of our own to determine just such a dream team.  The survey taught us a lot about the effect of a biased sample as our New York-based survey put Derek Jeter onto the team over Robin Yount, and Nomar Garciaparra only received one vote.  We suspect that had we taken the same poll in Boston, Nomar would make the team and Jeter would be left in the cold.

The tainted, biased results are as follows:

BILL DESIMONE is a lifelong Yankees fan, but in all other aspects of his
sporting interests, is a lover of the underdog.  Despite much recent evidence
to the contrary, Bill still believes all small market teams can compete
consistenly.  Bill is also in favor of expansion and realignment to fix
baseball’s woes.

DAVID PRINCE offers living proof that not every mediocre southpaw can find
a job in the majors, even if he is a college graduate.  He enjoys writing,
debating and facts of all kinds, although he has long suffered from
degenerative audience syndrome.  He cherishes wasting otherwise productive hot
summer nights listening to Yankee games on the radio and when he’s down on his
luck, he need only recall the Stump Merrill and Rich Kotite years to make
everything else look a lot brighter.