Sorry for the hiatus; sometimes a guy just has to go to Vegas (it was a lot of fun, thanks for asking). To get back into things, I’m going to follow up on my last post by looking at predictions made by Evan Z‘s ezPM model (there are updates, but that’s a good intro post). If you don’t remember, the idea was that predictive ability should be the hallmark of a model; if you can predict what is going to happen, you obviously understand what is going on. However, practically speaking, predictions vary for lots of reasons: model of productivity, aging curves, diminishing returns, predicting minutes played, etc. These differences make it very difficult to compare models in a reasonable manner. So I laid out a few very basic assumptions and presented prediction data using Wins Produced, the model of choice in the Skeptic household. I hoped that others would follow suit, and Evan generously did.
A quick word on ezPM (although you should really go to Evan’s blog for a full description and all the work that has gone into it). Similarly to WP and a number of other metrics, different events that happen on the court (made shot, missed free throw, etc) have a value at the team level. ezPM then breaks up these values to assign them to players, giving different credit depending on if a shot was assisted or not, what kind of rebound was gathered, etc. You add up all the stats a player acquires multiplied by their value and then adjust to per 100 possessions; this is the player’s ezPM100 value. Since the numbers come from play-by-play data, defense has been added by penalizing a player for what his counterpart does. Perhaps different from most other box score stats, ezPM is meant to reflect point differential as opposed to wins.
Since it’s based on play-by-play data with counterpart information, there are only one or two seasons of ezPM available. So to make predictions, only the current (ongoing) season was predicted. I followed the same method as I did for the WP post, with a couple of exceptions to accommodate ezPM. I started with the ezPM100 values from last season as the measure of a player’s productivity. It was assumed that every player this season would have the same production in however many possessions they played this season (the data is available on Evan’s site, and the numbers I used covered through Feb. 3). This honors the assumption that we know how much players will play so we don’t have to predict minutes. For players who were rookies this year or otherwise didn’t appear in last year’s data, they were assumed to have a production of -1.95 ezPM100, a value that Evan and I worked out in the comments on the last post. Then each player this season had a point differential predicted based on last year’s productivity and this year’s number of possessions used. The spreadsheet with the results is here (note that the ‘predwin’ column actually refers to point differential). Point differential was then summed up for each team (sheet 1 in that file; you can ignore 2010 since there’s no data to predict that year) and used to predict team wins so far this season. The prediction comes from an equation connecting point differential and win percentage (each point of differential is worth 3.28%, with 0 differential obviously equal to 50% win percentage); predicted win percentage was then multiplied by games played to get a predicted number of wins and compared to the teams’ actual number of games won. This is all on sheet 2, and the final measure is average absolute error. ezPM has an average error of 4.74 this year through Feb. 3.
Similar to the WP predictions, ezPM had a hard time with the Cavaliers. They are hugely underperforming, and the biggest offenders according to WP (in no particular order) are Hickson, Mo Williams, Jamison, and Moon. ezPM also thinks most of the Cavs have underperformed, with the most blame going to Hickson, Moon, Mo Williams, and Anthony Parker. So that’s a fair amount of overlap. In terms of who has actually done better than expected, WP picked Varejao, Sessions, Gibson (barely), Manny Harris, and Ryan Hollins (barely); ezPM picked Alonzo Gee, Hollins, Leon Powe, and Sessions. WP is not very happy with Gee this year but he’s only played about 300 minutes; ezPM is not very happy with Boobie Gibson, thinks Varejao took a tiny step back, and has Harris as being just under an average rookie. So a little more disagreement here but nothing too major. Overall, both WP and ezPM thought that Cleveland would have 20 wins instead of the 8 they had at the time.
The discussion in the comments from last post also inspired me to check a couple things on the WP predictions since they cover more years. One thing is that the WP predictions almost always predict more variance in wins than actually occurs. My guess is that this reflects the absence of any diminishing returns or regression to the mean in the model; players who have unusually good or bad years are expected to do the same the next year, and very good (or very bad) teams that add players don’t expect them to contribute less (or more) even though we know they will. Again, these are very simple predictions, so I don’t think it’s a big deal. If you run a regression predicting actual win percentage from WP predicted win percentage and use it to adjust the predictions for this season, the average error is virtually identical to ezPM’s.
So to sum up, it looks like for this short season (all the data is pre-All Star break), ezPM and WP make equally accurate and at least somewhat similar predictions. There certainly are differences, which should be expected since they use different weights on the box score stats. In general, it would be hard to make any big, sweeping conclusions anyway since there’s so little data. So hopefully this offseason I can get more data from more years for a variety of measures and put them all through the same exercise to get a good comparison. I’m sure someone out there has years of Win Shares (or if I have time I can download them), and I know that 5 year adjusted plus/minus numbers exist so there must be at least five years of that. If anyone would like to volunteer the data, or any other measures, I’ll be happy to run it.