Note: This section is now in read-only mode.
Please use our new community site for future posts.

Back To Board

Sample size not a problem this time

Posted By Fred Cline

Since batting average is itself a mean value, to say his sample size is 7 is incorrect. The sample size on all of those McGwire's is actually 7 x the 300+ ABs he has (presumably) had in each league. The correct SD for his BA would have to be computed from the data for each at-bat in each league, and the correct mean BA has to be computed by weighting the number of ABs that contributed to each BA. But SD isn't the best measure anyway.

The appropriate way to measure whether the McGwires in the SIM are statistically different from McGwire in real life is to compute the standard error of his BA in real life (not just the Standard Deviations) and the SE's for each of his simmed BAs. I think just by eyeballing it the fact his BA is lower by quite a bit in almost every league suggest that there will be a significant difference bewteen the SIM and real life McGwires when comparing the SEs. Plus the significant difference has direction, not just fluctuation.

In order to justify consistent directional significant differences between the SIM and real life you'd have evidence of significant differences in the situations - park, pitching, etc - if the SIM was working without problems. Since the SIM draws its stats from real life, over the course of a SIM season it would be unlikely for a player to have faced statistically significant differences in situations as they should average out, particularly accross multiple leagues. You can argue the McGwires could face better pitching and play in pitcher friendly parks against better defensive players - but enough to make a significant change in performance? Accross multiple leagues? Hard to justify.

Ultimately, the best way to measure this is how players did relative to each other and the league they were in. A McGwire that only hits 45 HRs (in his 70 HR season) would only be acceptable if the other players HR numbers were impacted accordingly. But if Sosa still hits 65, and Griffey hits 55, and so on, then that is a problem. You expect peak performance to regress to the mean some, but a top player should still be a top player relative to his league.

I don't think anyone is asking for an exact match between the SIM and real life, but to show my own example, I have a Mike Sirotka in Real Time 1 99 who has an ERA of 8.00 after close to 20 starts. In real life he is in the top five in ERA in the AL at under 4.00. Given that I am only two games out in the race, I figure a more accurate Sirotka, even one with and ERA of 6.00, probably wins me those two games. That is what frustrates people.

Fred Cline