A Quick Thought on Plus/Minus

The Wages of Wins guys have a post on plus/minus over at their blog, and Tango has something of a reply post over at his blog.  I thought I would throw in my two cents.

The WoW article weaves around a bit, but I think I can sum up the plus/minus part pretty succinctly: they think it does a bad job of telling you which players did well.  Not all of their points are right on-target, but the critical ones (with a tiny bit of my own expansion) are that: a) with plus/minus you can’t tell which player was specifically responsible for the team doing well in that time period; b) if you move to adjusted plus/minus to attempt to correct for that, you run into huge issues with sample size and colinearity; and c) the final results don’t seem to do a very good job of describing player quality.

Now, it’s important to note at this point that I don’t think many basketball stats guys would disagree with those three points.  Taking my usual reading habits (ESPN, WoW, APBR, a couple other things here and there) as representative, plus/minus and adjusted plus/minus simply aren’t used a lot.  When they are used, it’s almost always with a caveat, only to get a sense of defense (which isn’t necessarily captured well by the box score), or with other all-in-one metrics to get a more rounded view of a player.  A previous post of mine fleshes out some of the issues with APM and includes a number of links to other people describing and/or discussing it if you’re a curious reader.

So how about Tango?  Tango says that he likes APM for hockey (there isn’t enough scoring) but loves it for basketball.  As an aside, I imagine that if you ran your hockey APM regression using one of a logistic/mulitnomial/Poisson link, it would probably clean things up a bit.  But you would still have terrible colinearity issues, so I don’t think it would be a big help.  Moving on, plus/minus has a great intuitive hook in that it’s a with you/without you comparison; you typically run it with scoring but you could look at rebounding, fielding in baseball, whatever.  If a team has more of a good thing (or less of a bad thing) when a certain player is playing than when he isn’t playing, that’s a sign that the guy is doing something good.  Then it just becomes an issue of trying to tease out all the confounding info – the other guys on the field, who comes in to replace that player when he’s out, who the opponent has playing when he’s in versus out, and so on.  That’s why Tango says that you live for sample size: you need enough circumstances where a guy was both in and out with enough other stuff going on (various teammates, various opponents, etc) that you get a good sense of any given situation.

I think both camps here are right and there’s just a bit of miscommunication.  I wholeheartedly agree with Tango’s title: plus/minus is a GREAT concept.  And I also agree that if you have a big enough sample, it would probably do just fine.  The link he has to his catcher study sounds great.  The issue (and where I agree with the WoW article) is that, at least in the NBA, we rarely have the sample sizes needed to get APM to behave well.  And if you gather enough data to get it to behave well, now the interpretation of the number has changed quite a bit.  You can’t run a regression and scale things a bit to say “Kobe Bryant was worth 2 points a game to the Lakers in 2010″ with any confidence, but you might be able to say “Kobe Bryant was worth 2 points a game to the Lakers on average from 2005 to 2010″.  And then how useful is that?  If you wanted to know who the best player was over, say, a five year period, then I would tell you to fire up your APM calculations and have at it.  But that is rarely a question worth asking.

I want to know how good a player is now, or is likely to be next year, so that I can sign him to my team (whether I’m a real GM or a fantasy GM).  Even more helpful would be knowing why a player is good so that I can tell if he’ll mesh well with my system and the other guys on my team (if I’m a real GM; thank god that isn’t a fantasy issue).  APM doesn’t tell me that at all, although I suppose I could run regressions for any facet of the game I could think of (adjusted rebounding, adjusted assists, adjusted shooting, etc) and try to use that to create a picture of what a player contributes.  But the numbers APM gives me in any of those circumstances are going to be poor representations of the truth.  They won’t be useless; lots of stats out there have some value.  Player points per game tells you something about winning.  But there are better stats out there than APM, at least when basketball is concerned.

About these ads
This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

3 Responses to A Quick Thought on Plus/Minus

  1. Andrew says:

    I partially agree, but given the fact that the last version of RAPM a little better than WP at predicting results the next year than WP and as we’ll as ASPM or WS, is really interesting. It’s especially interesting because they’re not well correlated to each other.

    Daniel Meyers did a couple of interesting posts on the long term differences between RAPM and Win Shares, the first glance indicates that there maybe systematic undervaluation of defensive big men and a systematic overvaluation of offensive minded bigs. If that’s can be rigorously confirmed then it’s very, very useful.

    • Alex says:

      No doubt, RAPM does better than APM and potentially better than a lot of the boxscore metrics (I have some posts on that). But when it comes to straight plus/minus or APM, it’s hard to make a lot of positive claims.

      > Date: Sun, 17 Feb 2013 04:09:52 +0000 > To: akonkel@hotmail.com >

  2. Guy says:

    Alex: it seems to me that you are not in fact in agreement with the WOW camp. Their position is that +/- is a “bad stat” and nothing can be learned from it. The post specifically mentions RAPM, so they are aware of more sophisticated versions than simple 1-year APM. And both in the post and the comments they ridicule anyone who argues for using +/- even as a limited tool, to be used cautiously and in conjuction with other metrics efforts. Indeed, the whole point of the post seems to be that +/- must be removed from analysts’ toolkit — no other view is acceptable.

    In contrast, you seem to feel that RAPM may provide important insights, even if single-season APM has little or no value. That’s a very different perspective, and I suspect not that different from the outlook of most who use/develop +/- statistics. I would think that you might also agree that aggregated +/- data — across multiple seasons and multiple players — could be useful in evaluating and strengthening other metrics. To take your example, if “Kobe Bryant was worth 2 points a game to the Lakers on average from 2005 to 2010,″ that could potentially be very useful. If a boxscore metric said he was only average, that may indicate a flaw in that metric. And if that same metric undervalued SG more generally over those seasons, you almost certainly have an important finding.

    What separates the “camps” is not how valuable single-season APM is. It’s whether A) analysts should continue to work with +/- data in basketball, seeking out the signal within the noise, or B) analysts should stop using a “bad stat” and pretending it can have value. Unless I’m misreading you, I think you are still in the “A” camp.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s