My last post talked about the fact that unlikely things are bound to happen, and so we have to think carefully about what it means when something unusual happens. Then I was looking at Arturo’s tour of the WoW family of sites and saw that there was a post about Monta Ellis’ great game the other night. The upshot of it is that since Monta went 18 for 24, and under the binomial distribution that should only happen .27% of the time if Monta is a 44.9% shooter like he was last year, that perhaps Monta is a better shooter this year. And it’s possible that’s true. On the other hand, last year alone Monta also had games where he went 17 for 23 (p value = .0046 if he’s a 44.9% shooter) and 16 of 25 (p=.043), as well as games where he went 4 of 22 (p=.0085) and 2 of 14 (p=.017). So, within last year, Monta had games that were unlikely given how well he shot last year.

And you should expect this. Every distribution, including the binomial, has tails. Those are the unlikely parts that happen less than 5% of the time, or whatever threshold you want to use. Assuming he plays enough games, every player has his own distribution. And so every player should have games where he shoots much better than expected and much worse than expected. Michael Jordan was a career 49.7% shooter, essentially a coin-flip. It would be unlikely (p<.05) for him to hit 15 or more shots if he takes as many as 21 shots (e.g. go 15-15, 15-16, 15-17, up to 15-21; at 15-22 p=.06). But he had three games exactly in that range. You can find other unlikely games, like when he went 24 for 29 or 2 for 19. What do these games tell us? When they happen early in the year, they might make us worry that a shooter has lost his touch if it’s the 2 for 19 game; we might be optimistic that they got better if it’s the 24 for 29 game. But what we should really do is sit back, be happy about the win, and wait for more data to come in before we decide that someone has really changed how they play.

### Like this:

Like Loading...

*Related*

Alex, I pointed this out on my blog in reply to your comment, but I’ll reiterate here. If the p-level is 0.05, you would expect over the course of an 82 game season to have at least 4 games “beat the odds”. Even with p=0.01, you have a 56% of having 1 game during an 82-game season beat the odds. Essentially, you’re treating the per-game p-level as the probability of a successful trial.

The reason Monta’s outburst is so statistically improbable is simply because it happened with N=1 games played.

I think it’s a little more complicated than that, because it also depends on how many shots he takes. For example, if Monta only took 3 shots a game every game this year, it wouldn’t be possible for him to have an ‘outlier’ game because you can’t get to under p=.05 with only 3 trials. But that’s very unlikely, and you mentioned how the large number of shots he took make the game seem more reliable, which I agree with.

I’m also not sure that it’s fair to say it’s more impressive because n=1, because the n here is actually more than that; we’re basing our assumption on all of Monta’s games last year, and we also know he’s going to play another bunch of games this year. Another way we could frame it would be, given that Monta shot 44.9% on 1406 shots last year, how likely is he to have a group of shots where he goes 18 of 24? And to be honest, I would have to grab my stats book to check on how to calculate that, although faint outlines of a test are coming back to me.

I generally agree. I have this feeling, though, that we could work this all out to where we basically throw our hands up in the air and say “anything’s possible”. That wouldn’t be very helpful, though.

Also, doesn’t it make a difference whether we look at the i^th game vs. all games included in a certain range?

If we flip a coin and ask how many heads come up or how many runs of heads, that’s different than asking whether the 138th trial comes up heads. (I mean, I know that’s true, but I’m thinking something similar is going on with the Monta example.)

I think you’re right in terms of the “anything’s possible”, and that’s kind of what I was saying with my post. All we can really do is wait for more games to be played and see if Monta keeps shooting well (and I hope he does; he’s on my fantasy team). My last random unlikely stat: Ben Wallace once went 8 for 9 from the free throw line (p=.005 against his career average of .417).

I won’t belabor it, but if it’s useful for the binomial series you mentioned maybe doing: I don’t think it really matters which game we look at. We’re asking how likely it might be for someone to go 18 for 24 (to stick with Monta), but I don’t think there’s anything in the statistics about if those 24 are shots (trials) 1 to 24 or shots (trials) 1001 to 1024. It’s more of an issue of how many shots we have in the samples we want to compare; in this case, Monta’s shots last year or maybe in his career. The test I was thinking of is described here. I get a z score of 2.94, which is p = .0016, so we can still agree it was a very unlikely shooting night given his last season.

We’re mostly on the same page, but I want to make one point a little clearer. I’ll use your example. It’s true that the probability of shots 1-24 are the same as shots 1000-1024. However, what is different is the probability of *any* 24 shots vs. one particular series of 24 shots. To put it in terms of coins, the probability of getting 8/10 heads in flips 1-8, 100-108, or 1000-1008 are all the same and are each individually much, much lower than the cumulative probability of hitting heads 8/10 times in the first 1000 flips. You agree with that, right?

Sure, because in the second case the streak could be shots 1 to 10, 2 to 11, 3 to 12, etc up to 991 to 1000, so you have 990 chances to get that streak of 8 out of 10. But I’m not convinced that we should attach special importance to where in a sequence of flips the streak occurs.

Alex, let’s put it in terms of coin tossing. Let’s say a “game” = 10 coin tosses and a “season” has 82 games. What are the odds that a player gets 9 heads in a single game? What are the odds that during the course of a season a player gets 9 heads in a single game? Similarly, what are the odds that Monta hits 18/24 in a single game? What are the odds that Monta hits 18/24 in a single game over the course of the season? I think if you can answer these questions, you’ll be convinced.