My last post talked about the fact that unlikely things are bound to happen, and so we have to think carefully about what it means when something unusual happens. Then I was looking at Arturo’s tour of the WoW family of sites and saw that there was a post about Monta Ellis’ great game the other night. The upshot of it is that since Monta went 18 for 24, and under the binomial distribution that should only happen .27% of the time if Monta is a 44.9% shooter like he was last year, that perhaps Monta is a better shooter this year. And it’s possible that’s true. On the other hand, last year alone Monta also had games where he went 17 for 23 (p value = .0046 if he’s a 44.9% shooter) and 16 of 25 (p=.043), as well as games where he went 4 of 22 (p=.0085) and 2 of 14 (p=.017). So, within last year, Monta had games that were unlikely given how well he shot last year.
And you should expect this. Every distribution, including the binomial, has tails. Those are the unlikely parts that happen less than 5% of the time, or whatever threshold you want to use. Assuming he plays enough games, every player has his own distribution. And so every player should have games where he shoots much better than expected and much worse than expected. Michael Jordan was a career 49.7% shooter, essentially a coin-flip. It would be unlikely (p<.05) for him to hit 15 or more shots if he takes as many as 21 shots (e.g. go 15-15, 15-16, 15-17, up to 15-21; at 15-22 p=.06). But he had three games exactly in that range. You can find other unlikely games, like when he went 24 for 29 or 2 for 19. What do these games tell us? When they happen early in the year, they might make us worry that a shooter has lost his touch if it’s the 2 for 19 game; we might be optimistic that they got better if it’s the 24 for 29 game. But what we should really do is sit back, be happy about the win, and wait for more data to come in before we decide that someone has really changed how they play.