Eye Test vs. Math Test
A few weeks ago my friend Nick sent me a link to this epic 12-part series on Dennis Rodman’s basketball prowess. While Rodman has been in the news for some interesting reasons lately, prior to that he was a basketball player unlike any we’ve ever seen and this series sets out to prove the point. I was especially part of part 2(a)(i) on “Player Valuation and Conventional Wisdom,” which has a nice explanation on the battle between the eye-test and math-test in sports:
Yet chances are he remains skeptical of the crazy-talk he hears from the so-called “statistical experts” — and there is truth to this skepticism: a typical “fan” model is extremely flexible, takes many more variables from much more diverse data into account, and ultimately employs a very powerful neural network to arrive at its conclusions. Conversely, the “advanced” models are generally rigid, naïve, over-reaching, hubristic, prove much less than their creators believe, and claim even more. Models are to academics like screenplays are to Hollywood waiters: everyone has one, everyone thinks theirs is the best, and most of them are garbage. The broad reliability of “common sense” over time has earned it the benefit of the doubt, despite its high susceptibility to bias and its abundance of easily-provable errors.