Research Note: Tangotiger’s Pitch Count Estimators

Information to accompany Walt’s analysis.

In his recent article on seasonal pitch counts over time, Steve Treder used Tangotiger’s “basic” pitch count estimator (http://www.tangotiger.net/pitchCounts.html). Tangotiger has also introduced a more expanded version (http://www.tangotiger.net/pitchCountEstimator.html).

I want to address two issues here. First is how well these estimators work in predicting actual pitch counts for a moderate-sized sample of pitchers. Second is how these two estimators relate over the study period 1946-2003.

Empirical Verification

I have two small sets of seasonal pitch count data against which to verify these estimators. From ESPN, I downloaded seasonal pitch counts for qualified starters (i.e. at least 1 IP per team game played) in the years 2001 to 2003. Vinay Kumar provided me with summary data compiled from Retrosheet.org for all pitchers in the years 1988 to 1990. In this latter data set, there were a few PAs for most pitchers for which pitch information wasn’t available, so the pitch total was estimated assuming those missing PAs were average PAs for that pitcher.

The following table summarizes the quality of the fit of the two estimators:
2001-03
qualified 1988-90
qualified 1988-90
all
N 261 244 1,367

bPCE			

r (correlation)	.934	.951	.998

R2 (var. explained)	.872	.904	.996

Mean Error	+61.5	+177	+72

xPCE			

r (correlation)	.938	.959	.998

R2 (var. explained)	.879	.920	.996

Mean Error	-53.2	+41	+18

These correlate quite well with actual pitch counts. Still, at least when talking about qualified starters, there’s still 8-12% of the variance unaccounted for. Nevertheless, based on these small sample comparisons, the estimators should work well for noticing trends. The Spearman rank correlations are also quite high, so they appear well suited for ranking pitchers.

The mean errors are a little more troubling. They’re not huge and of relatively little concern for the xPCE, but are higher than we’d like for the bPCE. That’s still only about 2-5 pitches per start, but that may be the difference between eras.

I wondered if the mean error of the PCEs was stable for different pitch counts:

	2001-03

qualified	1988-90

qualified

N	261	244

BPCE		

<3,000 pitches	+86	+201

3,000-3,399	+55	+172

3,400+	+36	+144

xPCE		

<3,000 pitches	-24	+67

3,000-3,399	-61	+34

3,400+	-80	+9

There’s clearly a difference based on the number of pitches. Oddly, the raw number of pitches incorrectly estimated is higher with fewer pitches – note this is bias, not variance. This downward trend suggests that at higher level of pitch counts such as those in the 70s, both estimators might underestimate actual pitch counts.

In summary, these estimators appear to be fine for spotting trends across seasons and perhaps for ranking pitchers within a season. The xPCE appears to be better than the bPCE. But care should be taken when using individual pitch count estimates. The estimates appear to be biased, especially the bPCE which overestimates pitch counts consistently in these samples. Like any model, it may fit the sample quite well but still produce poor estimates for at least some individuals. Of course, these results are small sample results and these estimators should be tested against a larger sample.

Relationship over Time

While I don’t have the data to assess how well these estimators predict actual pitch counts over a long period of time, I can look at how they relate to one another over time. For each 10-year period between 1946 and 2003, I calculated the ratio of the bPCE to the xPCE. Remember, the years 1981, 1994, and 1995 are deleted due to the strike.

Year Ratio
1946-1955 1.053
1956-1965 1.042
1966-1975 1.042
1976-1985 1.049
1986-1995 1.042
1996-2003 1.035

Nothing to be too alarmed about, but the gap does vary by decade and perhaps by the pitch count, so using the bPCE probably overstates era differences.