Yesterday, a friend of mine e-mailed me a blog post he had read about Rudy Giuliani's SAT score, which was apparently 1073. The blogger questioned why we don't consider standardized tests like the SAT or IQ when selecting a president.
Reading that blog post reminded me just how hard it is to accurately measure human potential. Although the SAT and IQ can be important tools for measuring that potential, they are hardly the only tool one should use. The late Paul Wellstone had an SAT score in the 900 range, yet he want on to become a professor and a great Senator. Giuliani's SAT score was hardly breathtaking, but he went on to graduate magna cum laude from NYU law school, wipe out the NY mafia as U.S. Attorney, reduce crime as the NYC mayor, and heal the nation after 9/11. Another example of just how hard it is to predict human behavior: the average SAT score for the wealthiest 1% of the country is less than 1200 (the average of the top 1% of pre-1995 SAT scores was probably around 1500).
Measuring potential in sports, on the other hand, would seem to be much easier. You can measure athletic talent with a stopwatch, yardstick, or weight.
With that in mind, I decided to take a quick look today at last year's NFL Scouting Combine. As you might know, the combine is an invitation only event held each year by NFL scouts to take a look at prospects for the coming year's NFL draft. College players who are invited participate in a series of athletic tests, the results of which are published.
I decided to take a look at last year's wide receivers. I'm a football fan, not an expert, but I assumed that wideouts would benefit the most from raw athletic talent like speed and leaping ability.
Not every hopeful competes at all in the combine, especially if they know they will be drafted high (Santonio Holmes and Sinorice Moss skipped it in 2006). Those who do compete don't take part in every athletic competition.
Last year, there were still 31 wide receiver hopefuls who participated in the 40-yard dash, vertical jump, and broad jump skills competitions. Of those 31, 19 were drafted. I conducted an analysis of those 31 players by ranking them in athletic ability.
The method I used to rank them was relatively simple. I ranked them from 1-31 in each of the three categories, and turned those rankings into a percentile score (100 for first place, 0 for last). I then averaged each players results to come up with a composite score. I then ranked the players according to their composite scores, giving 100 to the best player and 0 to the worst.
Not surprisingly, the average undrafted player was in the 32nd percentile and the average drafted player was in the 61st. Oddly, however, the first half of draftees (as selected by teams) had a lower average percentile score -- 56 -- than the second half of draftees -- 67. Perhaps in later rounds the scouts are less familiar with players actual value and therefore more reliant on objective athletic measures?
So how did these draftees perform? Well, 10 of them didn't even make a single catch all season. Their average percentile was 57. The 9 who did make at least one catch had an average of 66. (Remember that these percentiles are based on comparisons with the 31 original combine participants.)
What's interesting is that 2 of the 12 undrafted players still made the NFL and made catches (Martin Nance and Hank Baskett). Nance had a 17th percentile score and Baskett, who had the third best year among all rookie wideouts who participated in the combine, had a 53.
Overall, the 11 wide receivers who both participated in the NFL Combine's 40 yard dash and jump competitions as well as got at least one reception averaged in the 61st percentile, a little above average. Overall, they averaged 276 yards.
4 received between 300 and 1038 yards. They averaged a 58th percentile score. 3 received between 100 and 300 yards. They averaged a 74th percentile score. 4 got less than 100 yards. They averaged a 53rd percentile score.
The top two receivers out of this group were Marques Colston (with 1,038 receiving yards and 8 TDs, Colston was the best rookie wide receiver and probably the best overall rookie) and Greg Jennings (632 yards, 3 TDs). Colston had a 70th percentile score and Jennings had a 45th percentile score.
Going back to the original 31 players I analyzed:
Marques Colston was clearly the steal of the draft as the 252nd pick overall. Even if he never catches another pass, his selection will go down as one of the best ever.
One thing that seems pretty clear in looking at these numbers is that it is more important to have sufficient athletic talent than superior athletic talent to perform well in the sport of football.
It would be silly of me to venture a guess as to how NFL scouts might be able to improve their model for predicting success. They get paid to do it and even they haven't figured it out. I suspect that they never will. Human beings are capable of doing the most surprising things. But even if scouts eventually get it down to an exact science, in tomorrow's draft they won't be able to do much more than take educated guesses and hope they get lucky with the next Marques Colston.
© Jed Lewison