With my new data set, I thought I would take a look at how the various metrics relate to each other. Please check that link for a description of the data set and links to all the sources. Obviously there are a lot of ways to do this, but here are some beginning shots.
The most obvious way to start is to check the correlations. Unfortunately not all of the numbers are available for everyone, so the overlap differs for different correlations. Still, the cor command for R has a useful feature called ‘use’, and you can set it to use any available pairwise values, even if a complete set isn’t there for the entire matrix. So the ezPM – old RAPM correlation is based only on 2010, but the ezPM – new RAPM correlation uses both 2010 and 2011, and so on. For everything in this post, I filtered out anyone who played fewer than 500 minutes for a team within a season. Unfortunately that cuts the data set by a third (I’m using the entire set I have of 10 years, and it goes from about 6000 player-team observations to about 4000), but hopefully the remaining data is more reliable. Anyway, here’s the matrix:
The ‘old’ and ‘new’ got cut off the row names and new RAPM got cut off from the last column; sorry about that. But here are a few observations. First, new RAPM (using the previous season as a prior) correlates at .917 with old RAPM (using 0 as a prior). So there isn’t a dramatic difference there. Similarly, old WP48 correlates at .916 with new WP48. They reported a correlation of .98; the difference is probably due to however I recombined WP48 values for players who were on multiple teams within a season for old WP. The correlation is obviously still high.
Did these changes affect how the metrics relate to other metrics? If you look at the WP48 and old WP48 columns, most of the correlations stay about the same. WP48 has a much lower correlation with PER than it used to, but nothing else changed by more than .04. RAPM, on the other hand, changed quite a bit. It now correlates at a higher rate with every other metric except for APM, which it already correlated fairly highly with, and ezPM (which still went up a little).
In general, the metrics roughly agree with each other. New RAPM correlates with everything with a value of at least .5, ezPM at least .5, APM at least .4, old RAPM at least .32, old WP at least .34 (and at least .72 with any boxscore-based measures), new WP .32/.57, Win Shares .39/.77, and PER .34/.57. But the correlations are only somewhat helpful; they take into account two variables at a time and we have eight.
We can expand on the correlations by seeing how well any one metric is predicted by all the others. I’m going to check this with regression. For example, I can predict a player’s PER from his WS48, WP48, RAPM, etc. Because the old and new versions of WP and RAPM are so highly correlated, I’m going to only use the new versions for this part. I’m also going to use scaled (mean centered and unit standard deviation) independent variables to get an idea of which metric is most predictive of the metric of interest. I’ll go in order of the variables in the matrix above.
PER can be predicted by Win Shares, Wins Produced, RAPM, APM, and ezPM with an R squared of .63. The biggest contributor is Win Shares followed by ezPM, although WS is nearly three times as important. WP is also a factor, although with a negative weight in this particular regression (remember that all the metrics are positively correlated; the weights and their signs will jump around depending on which ones you included in a particular regression). RAPM and APM are not big factors. Given that PER serves mostly as an indicator of popular opinion and is well-known to overvalue scoring, high correlations could be viewed as a negative, although I’ll try not to judge.
While ezPM is a significant predictor, there are only two seasons of it. Removing it increases the number of observations dramatically and didn’t end up affecting the R squared. RAPM and APM are now both significant, although slightly less important than WP (which is still negative). Win Shares is far and away the most important factor, with a weight about 8.5 times as large as WP. So essentially, the best way to predict PER is to know a player’s Win Shares, and players will do best if they do well on Win Shares but relatively poorly on Wins Produced. Of course, since Win Shares and WP are pretty well correlated, that’s a tough thing to find and a minor change.
Moving to Win Shares, the R squared from all the measures is .78; it’s more predictable than PER. PER and WP are the two biggest contributors and are about equal, although all the metrics are significant predictors (APM is negative). The spread is also smaller; PER and WP are about twice as important as RAPM, which is third. Again removing ezPM to increase the sample size, the R squared stays about the same (technically increasing to .79) and the other descriptions above stay the same.
Wins Produced is next. WP is predicted by the other metrics with an R squared of .746. The biggest contributors are ezPM and Win Shares with ezPM being a little more important. PER has a negative contribution in this regression, and APM and RAPM are virtually non-correlated. Again running the regression without ezPM, the R squared drops to .64. It looks like ezPM really has the most to say about WP (remember that the others so far haven’t really changed when it was removed). RAPM and APM are still virtually uncorrelated, Win Shares is far and away the most important predictor, and PER is negative. I was curious, so I checked: you can remove RAPM, APM, and PER and the R squared is basically unchanged from the original regression (.736) and the relative contribution of ezPM and Win Shares stays about the same. So those two alone are roughly sufficient to describe WP as well as it can be from these metrics.
Next is the first non-boxscore measure, RAPM. It is predicted by the other metrics with an R squared of .77. I found this somewhat surprising because I have a faint memory of boxscore stats not doing a great job of predicting APM (e.g. statistical plus/minus). Of course, it then turns out that APM is far and away the best predictor, nearly four times more important than Win Shares at number 2. ezPM is significant but a minor contributor, and PER and WP are non-significant. Thus it isn’t surprising that removing ezPM lowers the R squared a little (.736) but the rest of the story stays the same. For those who are curious, the boxscore measures alone only predict RAPM with an R squared of .398, led by ezPM and Win Shares; removing ezPM makes it .368 with roughly equal contributions from Win Shares and PER. So RAPM is fairly distinct from the boxscore measures after all.
Speaking of which, we’re on to APM. I don’t like APM, but I’ll cover it anyway. As you might have guessed, it is predicted fairly well (R squared of .727) and most strongly by RAPM. Removing ezPM drops the R squared to .6823 but removing RAPM drops it to .281. ezPM is then the biggest predictor, over two times as important as PER.
Finally we have ezPM, which is boxscore based but takes individual defense into account. If you were looking at broad strokes, it would be the next step in the move from WP (defense is only considered at the team level) to Win Shares (defense is part player, part team). ezPM is predicted by the other metrics with an R squared of .758. The biggest contributor is Wins Produced, over three times more important than PER. The other three metrics are pretty minor contributors. WP is a fairly important piece; the R squared drops to .638 if it is removed and Win Shares takes over as the biggest contributor.
So to sum up: each metric can be fairly well predicted if you know the other metrics, although RAPM and APM are distinct in that they predict each other pretty well but the boxscore measures contribute relatively little. That got me to thinking, maybe all these measures, although sometimes (usually?) viewed as competitors, are all saying roughly the same thing. To get a handle on that, I ran a principle components analysis. The short explanation is that a PCA turns your X variables (here, the eight metrics) into X uncorrelated variables (components) that are each combinations of the original variables. These new variables come in a particular order; the first accounts for as much of the variance in the original data as possible, the second the second most, and so on. Basically it’s a way to turn correlated variables into a smaller set (you don’t need to use all X variables) of uncorrelated variables that still describe the data fairly well. You can then look at the weights for creating the components to see where the variability in the data set is focused. An example from a class I took is race times for different countries in the Olympics; countries were observations (like players here) and races were the variables. Lower times in any race might be one component; relatively faster times on sprints as opposed to long races might be another. The interpretation is that countries generally vary most according to their speed in all the races, and next they vary most by if they tend to be better at sprints or long runs. The first component would separate your medal count contenders from your pretenders while the second would separate, for example, Caribbean countries from African countries. These components are independent though, so a country could score low on the first component (be generally slow) but high on the second (be better at sprints than long races) or really show any pattern.
In this case, the data set is limited to 2010-2011 since those are the only years of ezPM that exist. But the loadings are interesting; they basically follow the Olympic example I gave. The first component is to basically add equal amounts of each metric! That is, if you wanted to know if a player was good or not, the best first pass you could make would be to combine all eight measures in equal parts. That accounts for about 69% of the variance in player scores. The next component adds about 16% of the variance and is a contrast between the boxscore and non-boxscore metrics. Another 8% ignores RAPM and APM altogether and is a contrast between WP/ezPM and PER with a little Win Shares. I won’t describe the other components because the groupings become somewhat arbitrary and the variance described is low. But I think it’s interesting that if you wanted to describe how players vary as succinctly as possible, you would just see what the metrics say as a group.
As a kind of sanity check, I sorted the 2010 and 2011 players by their score on the first component. LeBron has entries 1 and 3 (3 was last year), with 2010 being the best score by a fair margin. Wade has 2 and 7 (7 was last year), with 2010 being a decent amount higher than LeBron’s 2011. LeBron’s 2010 is higher than Wade’s by about as much as LeBron’s 2011 is higher than the number 13 spot, which is Manu Ginobili’s 2010. Dwight Howard has 4 and 5, Chris Paul 6 and 9, Kevin Durant’s 2010 is 8th and Steve Nash’s 2010 rounds out the top 10. Scores start getting pretty tight by the time you hit number 11, so OKC supporters don’t need to be terribly offended that Durant apparently took a step back last year; he was still a top 30 guy across two seasons. About the only surprising name in the top 25 might be Greg Oden, who popped in at 16 for his 2010 ‘season’. I guess I should say surprising to me; others might still balk a bit at seeing Kevin Love, Nene, or Chris Andersen in a top 30 list. At the bottom of the list was Josh Powell (2010), Sasha Pavlovic (2010), Jonny Flynn (2011), Josh again, Mo Williams (2011), and Jannero Pargo (2010). Perhaps the ‘best’ player at the wrong end of the list is Aaron Brooks, although I’m sad to see 2011 Jason Maxiell come in at number 30.
For those of you who are curious about the RAPM/APM warriors on component 2, the top five are all from 2011 and are Nick Collison, Jason Collins, Ekpe Udoh, Ronnie Price, and Dirk Nowitzki; they all performed relatively better on RAPM/APM than the boxscore measures. The boxscore guys were Ed Davis (2011), Drew Gooden, J.J. Hickson, Nazr Mohammed, and Earl Boykins (all 2010). Finally, on component three we can see which guys are ‘stat nerd heroes’ (high on WP and ezPM but low on PER): Reggie Evans, Thabo Sefolosha, Shane Battier, Jeff Foster, and Ronnie Brewer, and which guys pass ‘the eye test': Andrea Bargnani, Amar’e Stoudemire, Marreese Speights, Brook Lopez, and Mo Williams. Speights is a surprise, but the top 30 is a virtual who’s who of NBA stars, which tells me that the component is doing what it claimed.
This was a fun exercise overall. I hope everyone found it as interesting as I did; I like the idea that we’re all roughly on the same page even if WP guys slam APM and RAPM and APBR guys slam WP and most everyone slams PER. It makes me feel that we’re really arguing about degree for the most part, and not so much about what to do in the first place. And I hope to get good mileage out of this data set in general. When I get a little time, I might try to do some player projections even though the season’s already started. Seems like a big project, but a data set like this may demand it.