The recent death of Paul the octopus and the start of the NBA season reminded me that it’s always good to think about what exactly statistics tell us about the world. If you don’t remember Paul, he correctly predicted all seven of Germany’s matches in this year’s World Cup as well as correctly picking Spain to win the final, thus going 8 for 8. According to Wikipedia, there is only a .39% chance of this happening if he was guessing randomly and thus expected him to be right 50% of the time (if you think a two-sided test would be better, it becomes .78%). If you include the fact that the first few matches could have ended in ties, the percentage would be even smaller. Under typical statistical guidelines, you would reject the hypothesis that Paul was working from 50%; you’d think that he knew what he was doing.
But, I think we can all agree that the octopus had no special knowledge of soccer games. And so what we have here is a case where we have to think about what the null hypothesis means exactly. In this case the null is that the probability of Paul picking a winner is 50%. Observing him make correct predictions 8 out of 8 tries should only happen a fraction of a percent of the time (it’s equally likely that he could have gone 0 of 8, by the way). But, it should happen sometimes. Similarly, the winning lottery numbers in Israel recently repeated themselves within a few weeks. This should only happen once in 10,000 years according to a stats professor. But, obviously it has happened.
Again, under most hypothesis testing situations, we would reject the idea that Paul was picking at chance or that the Israeli lotto wasn’t rigged. And again, we would be wrong (it’s called a type I error). I think it’s a nice reminder for those of us who take statistics and testing seriously that sometimes you’re going to have a significant result that isn’t real. Similarly, I think it’s a good reminder for non-stats-minded folks to be reminded that sometimes unlikely things happen. I think that as humans, we’re driven to look for causes and explanations for things. We want to attach meaning to events, even when we don’t have a lot of information, and sometimes we go so far as to assume that something mystical must have happened, like the existence of a precognitive octopus. Instead we should realize that sometimes the unlikely occurs.
At the beginning of the season, people will try to attach importance to certain games, like Boston beating the Heat. As the season goes on, people will focus on losses to bad teams or potential playoff match-ups. But sometimes a bad team will beat a good team; if two good teams (who are by definition relatively evenly matched) play each other, someone has to win. This is why the regular season for most sports is so long; you need a lot of games to really tell who the good teams are. This is why football, and the playoffs, are frustrating (you can read a good playoffs quote from The Drunkard’s Walk here), because you have to try to attach importance to every game because there are so few available for deciding who’s best. In the end, we would all do better to sit back once in a while and say hey, things happen.