The Challenge

Guy offered a challenge to me recently which I accepted.  It probably wasn’t on par with The Contest or the Battle of Wits (have you heard of Hollinger? Berri? Ilardi?  Morons!), but it was fun nonetheless.  The challenge was this: Guy would predict player WP48 using their position and rebounds per 48 minutes (R48) while I would predict it using their position and true shooting percentage (TS%).  The idea is that if Guy won, it would show that Wins Produced overvalues rebounding.  If I won, it would show that it’s perfectly ok (maybe not).  The gory details are below, but here above the cut, for all to see, congrats to Guy.  But for WP fans, don’t worry; there’s plenty of equivocation below. 

Here are the details.  I have two seasons (2008 and 2009) of player data from basketball-reference.com that I paired with WP48 and position info from the automated site.  I suggested cutting out all the players who had a mixed position listed just for ease of processing, and Guy suggested cutting out anyone who played fewer than 1000 minutes to ensure we were using good estimates (and also because including the low-minute players resulted in weird models, although I didn’t ask what made them weird).  This left us with 219 player-seasons for our sample out of a pool of about 880.

Going in, I expected to win because I ran my regression on a bigger set and found that TS% beat out R48.  But, in our final sample, R48 did indeed predict WP48 better.  Since our dependent measure is the same, we can simply compare R squared values, and TS% has .295 while R48 has .482.  I also checked points per shot (PPS) as another measure of shooting prowess (TS% includes an adjustment for free throws taken while PPS does not) and it had an R squared of .304.  So Guy won the bet, and congrats to him again, but I was confused.  What changed the order of importance of the variables from when I took the bet?  I’ll start at the top and work my way down.

If I use the full sample and run our same regressions (position along with one of R48, TS%, or PPS), the R squared values (in that order) are .629, .541, and .421.  If I leave position out, the values are .26, .471, and .352.  So including position in the model makes a difference; if you didn’t, you would assume that shooting were better predictors than rebounding.

What if I use the full sample but take out all the mixed position players?  Zach Randolph, for example, is listed as part power forward and part center.  I suggested taking guys like this out for computational ease, although it turns out that I didn’t need to (R is amazing!).  It doesn’t make a big difference; the R48, TS%, and PPS (along with position) R squared values are .645, .51, and .354; without position they are .316, .503, and .342.  So we have the same story as we did in the full sample.

What if I take out players with less than 1000 minutes but include mixed position players?  I get an order of .679, .503, and .503.  Shooting takes precedence again if I remove position; the order is .173, .233, and .248.  And we know what happens if I take out mixed position players since that was the challenge; if position was removed from the challenge, the order is .126, .280, and .277 and I would have come out on top.

Choosing 1000 minutes as a cut-off was arbitrary; does it matter if I use a different criterion?  I took the full sample and cut out players who averaged less than ten minutes per game.  The results are much closer; .565, .520, .513 (and of course shooting comes out ahead if position is removed from each model).  Removing the mixed position players results in the same pattern.  So had the challenge used a different minute cut-off Guy still would have won, but it would have been much closer.

Summary time: using a different minutes-played criterion does make a difference, although not in the final result; however, there’s no ‘best’ way to pick how many minutes a guy should play before we think his stats are reasonably well-estimated.  I thought that taking out mixed position players would make analysis easier, although it didn’t matter in the end for either the analysis or the results (although I don’t know what program Guy uses; perhaps it was helpful for him).  Instead, the key factor was including position in the models.  In the full sample position alone only had an R squared of .1 when predicting WP48 and the regression was not significant, which makes sense since WP48 is standardized to position (the R squared is virtually 0 if mixed position players are removed).  If you don’t include position, which is how I think I did my initial check when taking the bet, shooting is a better predictor of WP48 than rebounding.  But when position is included, rebounding flies past.

This is odd given that I just said that position barely matters; however, rebounding is fairly correlated with position.  Position doesn’t predict TS% at all; the R squared is .157 but not significant.  In contrast, position predicts R48 with an R squared of .576.  And now we’ve come full-circle back to a point I made in my post on missing variables; when variables are collinear, even ones you haven’t put in the model yet, there is a high potential for changing things drastically when you move from model to model.  Position or rebounding alone are fairly poor predictors of WP48, but combined they appear to be great.  Adding position on top of true shooting raises the R squared, but not a lot.  Another warning sign is that the value of rebounds changes drastically depending on if position is included or not.  Without position in the model each extra rebound per minute buys you .02 WP48; with position, it doubles to .043.  This isn’t due to any kind of interaction, like rebounds count more for centers; the model now thinks that rebounding is more important than it did before.  You get a similar effect with assists per 48 minutes because it also correlates with position; the value of an extra assist per 48 minutes doubles if position is in the model.  You can also see the collinearity effect if you compare a model with both TS% and R48 (both standardized to allow for comparison) to a model with those plus position.  If position is included R48 has the higher standardized coefficient and seems more important; if not, TS% has the higher coefficient and seems more important.

This is one of the reasons for the quote popularized by Twain: “there are three kinds of lies: lies, damned lies, and statistics”.  A lie can be figured out; you can sometimes tell when someone is being sneaky with a damned lie.  But with statistics, sometimes it’s hard to tell what’s going on at all.  Different analyses, all equally valid, can give you different answers.  Changing your minutes played cut-off changes things a little; the seemingly innocuous decision to include position or not changes things a lot.  So in the question of which is more important to WP48 (or PER, or adjusted +/-, or team wins), rebounding or shooting, it’s going to depend on what else is in the model.  I think the fairest answer is that they’re probably about equally important.  But I’m sure the debate will continue unabated.

 

About these ads
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

48 Responses to The Challenge

  1. Guy says:

    Hey Alex: I don’t believe the 1,000 MP cutoff made any difference. If I had given you my model for all the players in your data set, the R^2 increases to .64, much higher than you report for your models.

    Position does matter, of course, for the simple reason that WP uses a position adjustment. This position adjustment is pretty close to the average Reb48 for each position, so it effectively position-adjusts rebounds. The reason the R^2 drops a lot if you remove position (in a reb48 model) is just that the model then can’t tell if a 9.0 Reb48 is really good (PG) or really bad (C). That same adjustment has virtually no effect on shooting efficiency. And if Wins Produced used the same adjustment at all positions to convert AdjP48 into WP48, then Reb48 could predict WP48 very well without a position adjustment.

    So this doesn’t change at all the conclusion that rebounds is the main driver of WP48. Rebounds determine about half of the variance in WP48, and much more than shooting efficiency. In other words, Wins Produced says that rebounds are as important as shooting efficiency, turnovers, FT, PF, blocks, and assists — COMBINED. And that would be fine, except for one thing: Wins Produced at the team level tells us that rebounds really account for only about 10-15% of wins in the NBA. And so WP48 cannot possibly be giving us an accurate read on productivity at the player level.

  2. Guy says:

    I need to correct a mistake in the last paragraph of my last comment. Reb48 alone does explain about half of the variance in WP48, but that’s in part because it is correlated with other forms of productivity (as is shooting efficiency). In a fully-specified model, reb48 would explain a smaller proportion of the variance, and thus is not as powerful as all other variables combined. However, it remains true that:
    * rebounds is the single strongest determining factor of WP;
    * rebounds has more impact on player WP than shooting efficiency (by about 25%);
    * rebounds are actually much less important than shooting efficiency in terms of creating real wins in the NBA.

    It’s also apparent looking at the full model that one reason for the strong correlation between rebounds and WP is that shooting efficiency actually receives too little weight.

    • Alex says:

      If by shooting efficiency you mean true shooting, rebounding is far more correlated with the other productivity stats than shooting efficiency. That’s a bet I’ll take any day of the week.

      I’m curious Guy, what’s your opinion of Win Shares?

  3. ilikeflowers says:

    Alex,

    Assuming that two independent variables have in reality equal contributions towards some third value, but one (A) has twice the variation of the other (B), shouldn’t the result of this kind of analysis (a single run on the entire sample?) be expected as something like:

    small sample size – A has less predictive power than B
    medium sample size – A has somewhat less predictive power than B
    large sample size – A has the same predictive power as B

    To eliminate this issue don’t you need to run this analysis on random sub samples and average out the results (you may have and I just missed it), or examine the trend when going from small sample size to large (using the final sample set data criteria and perhaps a larger sample as well)?

    • Alex says:

      We went through some of this with the discussion with Phil. What you said should be generally true, but need not always be true in any particular sub-sample. I don’t think averaging random sub-samples is necessary; you can just use the full sample. I ran both of our models on a variety of subsets of the data (with and without mixed position players, with and without players who played under 1000 minutes, with and without players who played under 10 minutes a game), and none of it changed the pattern substantially. The cross-validation you suggest would be more helpful if we used vastly different models, but I don’t think it’s too much of an issue here.

      • ilikeflowers says:

        I was mainly thinking that the 219 player seasons could be too small of a sample size here and that examining random subsets of those 219 player seasons using increasing subset sizes might show a trend towards convergence or crossing at larger sample sizes constructed using the same criteria as the 219 player season sample. This scenario would be consistent with standardized coefficients saying the order is A, B and elasticity saying the opposite.

      • ilikeflowers says:

        Disregard the last sentence in my previous post. It’s incorrectly mixing observations from different samples and is meaningless.

  4. Guy says:

    The whole point of the exercise was to demonstrate that elasticities will not, and cannot, tell us how large a role rebounds (or other variables) play in determining players’ WP. That is not the question they are designed to answer. Elasticities are basically another way to report regression coefficients — they tell you how much Y will change when X changes a specified amount (1%). But that’s only half the information you need to determine how big a role X plays in determining Y. The other half is the variance in X. If X doesn’t vary much, even a large coefficient won’t allow X to have much influence on Y; if X varies a lot, the influence can be large even with a modest coefficient. To take an extreme example, if I build a model to predict reb48 and use positions as predictor variables, the I can generate a coefficient (and elasticity) for “Center.” But obviously the total influence on team rebounds is zero, because every team has a center (variance = zero).

    So you need to know both the impact of a one unit change in the predictor variable AND how much variation there is in the variable. It’s the product of the coefficient and the variance that matters. So that’s why you want to use standardized coefficients — they tell you how much of the variance in Y is accounted for by X.

    It’s also important to note that if we repeated this exercise but tried to predict team WP, I would lose that bet badly. At the team level, knowing PPS will give you more predictive power than knowing rebounds — because shooting efficiency really plays a larger role in determining wins. Unfortunately, at the player level WP has the relative importance of these factors upside down.

    • ilikeflowers says:

      The whole point of the exercise was to demonstrate that elasticities will not, and cannot, tell us how large a role rebounds (or other variables) play in determining players’ WP.

      If that is your goal then we need to (at the very least) see the elasticities and standardized coefficients for the actual 219 sample used here (and at best use a bigger sample).

      Also, this from Alex Going in, I expected to win because I ran my regression on a bigger set and found that TS% beat out R48 argues in favor of the wow FAQ elasticity results.

      • ilikeflowers says:

        Just to clarify, it’s not specifically the 219 sample size that is the issue. It’s the difference between the selection criteria for the data in the FAQ and the selection criteria in the 219 sample.

    • Guy says:

      Flowers, the debate is over. Empirically, rebounds has a larger impact on — i.e. determines more of the variation in — players’ WP48 than does PPS. There is no way my R^2 can be so much larger than Alex’s unless rebounds can better predict WP48. And theoretically, we KNOW that elasticities are the wrong tool and cannot provide the answer, because they tell you nothing about how much players actually differ in rebounding or PPS. It is literally impossible to anwer this question based on elasticities alone. It would be like claiming that Reb48 can tells us which player on a team had the most rebounds in a season, without knowing MP.

      Sample size won’t materially change anything, because WP48 is a formula based (in part) on these variables — it’s not like trying to tease out the relationship between race or poverty or something like that. Alex’s report of earlier success was not based on a larger sample. Read his post again: he neglected to include position in his model. If you don’t adjust rebounds for position, as WP does, then it’s true their predictive power declines.

      Rather than looking frantically for a reason to ignore this result, you might better ask why Dr. Berri trotted out these largely irrelevant elasticities in the first place.

      • ilikeflowers says:

        You got in before my clarification. Reference it.

        Nothing frantic here. I find this topic interesting and my natural tendency is to identify flaws. As always I prefer to make conclusions about oranges based upon evidence about oranges.

      • ilikeflowers says:

        Rather than looking frantically for a reason to ignore this result, you might better ask why Dr. Berri trotted out these largely irrelevant elasticities in the first place.

        There’s no need, I already looked into it.

        After your elasticity/Berri rant at courtsideanalyst I did a ten minute Google search on ‘elasticity vs standardized coefficients’ and some other related searches (adding ‘vs’ and/or ‘regression’). I suspected that it might be more of a religious war among the stat heads.

        For my efforts I saw that standardized coefficients:
        [1] are not applicable when comparing across populations
        [2] were the standard prior to about 1990 after which its popularity began to decline
        [3] have some theoretical issues related to assumptions about the appropriateness of comparing the standard deviations of variables that may be in different units.

        I didn’t find any objections to the use of elasticity instead of standardized coefficients (so it doesn’t appear to be a religious war).

        I did find objections and limitations as to the use of standardized coefficients that are not applicable to elasticity.

        The most favorable position on standardized coefficients that I found simply stated that sometimes one was better than the other without providing any further details

        So here I am, a non-expert in this field evaluating:

        [a] Respected professor’s opinion that elasticities are perfectly valid in this case
        [b] 10 links on the point from lectures and books by experts (presumably, I’m playing the odds here) that indicate that elasticities are more broadly applicable and at least as good to use as standardized coefficients.
        [c] The opinion of a random internet poster who claims to know the truth about oranges based upon evidence about apples.
        [d] A challenge about a specific instance comparing results from a filtered dataset to those from an unfiltered(?) dataset.

        The logical conclusion from this is that the random internet poster is incorrect and that elasticities are (at least) as valid to use as standardized coefficients for determining relative importance.

        Even if I strengthen c and d to:
        [c] The opinion of a respected internet personage
        [d] A conclusive challenge about a specific instance

        The logical conclusion remains the same.

        What I take from all of this is that:
        [1] where possible, one should use both as a general rule
        [2] when they disagree (significantly) there is an opportunity to gain further insight into the nature of a specific population
        [3] if you can only use either elasticity or standardized coefficients (and you don’t know the units) then you should use elasticity since it: is more broadly applicable, is more intuitive, has no units issues, has no standard deviation comparison concerns.

      • Guy says:

        Flowers: You could make this decision based on credentials (or rather, your assumption about credentials), or based on the average result of 10 random Google searches (LOL). I guess you’ve ruled out an approach some people might take, which is trying to understand the statistical principles involved and make an informed judgment. But if you are really so confident in Dr. Berri and/or Dr. Google, how about you and I make the same bet? You download WP from Dre’s site for any season(s) you want, and append both Reb48 and PPS (Pts-FT/FGA). I’ve got $100 that says I can predict WP48 better than you can, using the same process that Alex and I used.

        Just to be clear, I don’t recommend that you take this bet, as you will lose and I don’t want to take your money. I’m just hoping this will motivate you to accept reality.

      • ilikeflowers says:

        Guy, it seems like you are failing to differentiate between apples and oranges again. Based upon this rant:

        These elasticities tell us very little about how big a role rebounds play in determining players’ WP48. For that purpose, one uses standardized coefficients not elasticities…And I have to say it’s a bit unkind of you to reproduce this particular post of Dr. Berri’s, which makes him look like a statistical illiterate. I’m sure he’s rather embarrassed about it at this point. So I think we should just forget about this mistaken use of elasticities (and also his confusion about when to use coefficient of variation), and focus on the real issues.

        and this

        Rather than looking frantically for a reason to ignore this result, you might better ask why Dr. Berri trotted out these largely irrelevant elasticities in the first place.

        It appears that a fair interpretation of your meaning is:

        ‘When determining how big a role one independent variable has compared to another to use elasticity instead of standardized coefficients is to make one look like a statistical illiterate.’

        Not in any specific case, but in general. Is this a fair assessment of the meaning in your posts? I thought that it was clear that this is what my previous post was concerning. If this is not a fair assessment then please disregard the rest of this post.

        Back to the task. A trivial amount of investigation reveals that this statement:

        ‘When determining how big a role one independent variable has compared to another to use elasticity instead of standardized coefficients is to make one look like a statistical illiterate.’

        and others with it’s same meaning are nonsense.

        I guess you’ve ruled out an approach some people might take, which is trying to understand the statistical principles involved and make an informed judgment.

        Already did it. The statistical principles involved in both standardized coefficients and elasticities appear straightforward. The objections I found to standardized coefficients were concrete, understandable, and trivially easy to location. If the issue were as your hyperbole indicates, then it would be comparably trivial to find support for your position given the widespread use of elasticity.

        I’ll attempt to post the links again if necessary, but I can’t stress enough how trivial it is to find them. I encourage anyone who’s interested to look into it for yourself instead of bothering with the opinions of two internet nobodies. You will quickly find relevant information in statistics books, professional conference presentations, specific examples of when elasticities are preferable to standardized coefficients, and theoretical concerns.

        But if you are really so confident in Dr. Berri and/or Dr. Google, how about you and I make the same bet?

        What I am confident in is listed at the end of my previous post. I am also confident that if two independent variables have roughly the same units then standardized coefficients are indeed preferable when dealing with a specific dataset. I am not confident about the relative merits one way or another regarding elasticity vs standardized coefficients involving the dataset used in this challenge and the dataset used in the wow FAQ.

        I’ve got $100 that says I can predict WP48 better than you can, using the same process that Alex and I used.

        This is a strawman. The issue is the relative merits of both in general, not in any specific case. Else why would using elasticity make one a “statistical illiterate”?

      • Guy says:

        That’s what I figured. Bluff called.

      • ilikeflowers says:

        That’s what I figured. Point missed.

        When to use elasticity and when to use standardized coefficients
        When to use elasticity and when to use standardized coefficients. More details from the same source.
        When to use elasticity and when to use standardized coefficients
        Describes both
        Describes both
        Describes both and notes that choosing one over the other is generally cultural unless there is a specific theoretical reason to pick one.
        Talks about the benefits and drawbacks of both

        That’s seven links (six sources) from the first two pages of searching Google for “elasticity standardized coefficients”. I could provide more but there’s really no point. I’ve left no relevant links out.

        My take from these sources is that unless you have some specific theoretical concern or need to compare results across datasets use whichever you like (and preferably both). If you need to compare results across datasets definitely use elasticity. Otherwise, it depends upon the nature of the relationships in your dataset which means that you probably need to run both and look into it more if they disagree significantly. Par for the course with statistics as Alex notes.

        So clearly these statements from Guy:

        These elasticities tell us very little about how big a role rebounds play in determining players’ WP48. For that purpose, one uses standardized coefficients not elasticities. And I have to say it’s a bit unkind of you to reproduce this particular post of Dr. Berri’s, which makes him look like a statistical illiterate. I’m sure he’s rather embarrassed about it at this point. So I think we should just forget about this mistaken use of elasticities (and also his confusion about when to use coefficient of variation), and focus on the real issues.

        and

        …you might better ask why Dr. Berri trotted out these largely irrelevant elasticities in the first place.

        are silly nonsense.

        As for this specific 219 dataset (Alex, btw what are the standardized coefficients and elasticities for the various datasets?) for the challenge, I’m leaning towards RB48 being best if you only use it (the apples). However as is evident from my ‘research’ and Alex’s analysis of this specific case it’s never that simple, especially when you start adding multiple independent variables (the reality) into the predictions and the interactions start to emerge which can change the relative importance (depending upon the nature of the data).

        There’s nothing more for me to say on the matter so I’ll just repeat some sage advice I once heard,

        …when a smart guy…says something that strikes you as a silly, elementary error, consider that an alternative explanation is much, much more likely…if you accuse them of doing something extremely foolish, it’s not they who will usually end up looking the fool.

        Guy 2011

      • ilikeflowers says:

        Alex, ignore my question to you in my previous post (still awaiting moderation). It’s too much work and probably not even relevant.

        • Alex says:

          There are a lot of potential models, you’re right. If there are one or two you’d really like I can post them. I’m sure you saw in your links that you can get elasticity from the regression equation, so I would leave that to you, especially since it depends on your choice of x and y values.

      • ilikeflowers says:

        The entire subject of elasticity vs standardized coefficients, linear vs non-linear effects, and hidden/missing independent variables is very interesting. I’ve read about techniques for dealing with non-linearity, but it’d be interesting to find out if one can use both elasticity and standardized coefficients to tease out useful info about the actual interactions/missing variables in a dataset. Maybe by plotting them while increasing sample size and rerunning the regression each time. Fascinating stuff. I’d love to look into it myself but unfortunately, NBA stat-hacking is my second favorite hobby and I am utterly devoted to my first.

    • Phil Birnbaum says:

      Here’s an extreme but stupid example that illustrates the point. Suppose you want to predict how often a Hollywood star gets his picture in the paper. (Assume there are 1,000 stars.)

      You determine that the more interviews a star gives, the more pictures appear, in a 1:1 ratio. So the elasticity is 1. Fifty percent more interviews means fifty percent more pictures.

      Now, you look at number of eyes. There’s one star with three eyes, and he has thousands of pictures all over the place. The elasticity is 100. Fifty percent more eyes means 5,000 percent more pictures.

      But which would you rather know when trying to predict how many pictures? You’d rather have number of interviews. Because, if you choose number of eyes, 999 times out of 1,000, your prediction won’t improve much at all. Three-eyed stars are so rare that knowing the number of eyes doesn’t give you as much useful information as knowing the number of interviews — even though the elasticity is much, much greater.

      This is what Guy is trying to say. It’s not enough to know the elasticity; you have to know the variance, too. Eyes have high elasticity but low variance, so over the entire sample, “eyes” in general doesn’t tell you much.

      Sorry the example is so stupid, but it illustrates the point.

  5. ilikeflowers says:

    wordpress won’t accept my post with the actual links, but if one merely searches google for ‘elasticity standardized coefficients’ and spends a few minutes on the first page or two you’ll find:

    3 links that can be summarized as “Cases in which standardized coefficients can be misleading, elasticities sometimes better”
    1 link as “Standardized coefficients describe relative importance, elasticities describe percent change”.
    1 link “Sometimes one, sometimes the other is better”

  6. Hey Alex,
    Cool post. So R48 and Position interact positively with each other but TS% and position don’t. How about you throw in Usage or FGA48 (ask dre for the data)? Betcha it makes a huge difference in the test. I’m thinking Reb48+Position is weaker than TS% and Usage (or FGA48)

    • Alex says:

      I can do FGA48 pretty easily; it has nothing to do with wp48 either, so it doesn’t add any predictive power on top of TS%. Usage might be different, but I wouldn’t count on it too much. I’m not sure if it would do much more than stand in for position, although that’s just my intuition, and adding position to TS% doesn’t help much at all.

      I think it has more to do with the intercorrelations of the variables; TS%, rebounds, assists, blocks, steals, fouls, and turnovers are fairly unrelated on their own but as soon as you know position you can predict rebounding 60% better, assists 88% better, steals 19% better, blocks 50% better, turnovers 32% better, fouls 28% better, and TS% 20% better. I’m not sure how many of those are significant, but obviously assists, rebounds, and blocks gain a lot. Those are the stats that really vary across positions (at the per minute level at least). Assists and blocks aren’t worth as much, so they don’t predict wp48 as well.

  7. Pingback: Rating the Centers | The City

  8. guy says:

    I fear that simultaneously confronting “Berri wrong” and “Guy right” has overloaded flowers’ wiring and fried his brain. Hopefully, with time, he will recover. In the unlikely event anyone has made it this far, the issue isn’t that complicated. Elasticities are perfectly valid tools, of course. They simply can’t answer THIS specific question, which is how much of players’ WP is determined by their rebounding and how much by their FG shooting efficiency?

    By the way, be sure to check out Phil Birnbaum’s last two posts at Sabermetric Research, which provide overwhelming evidence of large diminishing returns on Drebs, and some DR on Orebs as well.

  9. EntityAbyss says:

    There’s something wrong with this that I noticed. In this, you only considered a player’s scoring efficiency rather than his entire scoring ability. WP isn’t just about scoring efficiency as it’s also not just about scoring. It’s about both (in regards to scoring). The way you did it, of course rebounding would have a higher correlation, because scoring isn’t just about scoring efficiency.

    Someone who averages 10 points, but has a high ts% shouldn’t expect to improve his WP by much. Scoring wins are about scoring efficiency AND the amount of points you score.

    Also, outside of that, it’s mostly about possessions as a whole, not just rebounding. Most people don’t see net possessions so it seems like rebounding is a huge factor. The thing that gets missed out is that most players who are thought of as great players by common perception tend to average around 3 turnovers and around 1 steal a game. SO the majority of players average 2 less net possessions than their rebounding totals. Players like Marcus Camby and Ben Wallace don’t have much of a difference between steals and turnovers, so they gain more net possessions. I think that goes unnoticed. Check that out.

  10. EntityAbyss says:

    Sorry for the double post, but to illustrate my point is Andres’ numbers last year on 2010′s top role players. If you look at the scorer role numbers (which is scoring efficiency AND scoring totals) and the defender roles (which is mostly rebounding (and doesn’t consider loss of possessions due to turnovers or gain with steals)), you’ll see that numbers are very similar. scoring (in its entirety and not just efficiency) and possession factors have very close effects on WP.

    http://nerdnumbers.com/archives/69

    • Alex says:

      That’s a good point; I forgot about Dre’s breakdown (he should be mad at me).

    • Guy says:

      No, nothing EA says supports the conclusion that our exercise was “wrong.” First, let’s remember where we started: Alex’s claim, following Dr. Berri’s FAQ, that scoring efficiency plays a larger role than rebounds in determing players’ WP48. Berri asks “What is the role rebounds play in a player’s WP48?” and provides these elasticities as his answer:
      Points per field goal attempt: 5.2%
      Rebounds: 3.2%
      Free throw percentage: 1.2%
      Personal fouls: -1.1%
      Assists: 1%
      Turnovers: -0.9%
      Steals: 0.7%
      Blocked shots: 0.2%

      This is intended to show that shooting efficiency plays a larger role than rebounds (or any other single variable) in determining WP. That claim is false, as I demonstrated by my ability to predict WP more accurately using Reb48 than Alex could do with PPS. I could do that because elasticities are the wrong way to determine how much of the variance in WP each factor accounts for. (And it would be nice if Alex would confirm that, for ilikeflowers’ benefit.)

      Second, as it happens I did the same exercise replacing PPS with “net FG points” (Points – FT – FGA – .5*FTA), which would take care of EA’s objection that you need to account for both efficiency and usage together. It makes little difference: net points is still a weaker predictor of WP than Reb48.
      (And btw, assists play a huge role in determining WP48, despite the 1% elasticity — another indication that elasticities are irrelevant here).

      And of course the discussion was never about “possessions overall.” While TO’s should probably be adjusted for a player’s usage, I have no serious quarrel with the way WP values TO or STL.

      If you want to make a correct criticism of this exercise, let me help you out: it is that Alex and I looked only at PPS and rebounds, rather than a full model with all relevant variables. It’s possible that Reb48 only appears to be a strong predictor of WP48 because it correlates with other factors (such as blocks). However, it turns out that when you do a full model, Reb48 STILL accounts for more of the variance in WP than does PPS, and is still the single largest factor driving WP. And so the ending of the story remains the same: Berri’s elasticities do not answer the question at hand, and WP does overvalue rebounds.

      • Alex says:

        To be clear, I’m fine with how the wager went; we picked conditions and stuck to them. But I haven’t come to a conclusion about if standardized coefficients or elasticities are ‘best’ in this case. One of ILF’s links had an equation that linked coefficients and elasticity, so it isn’t clear to me that one has to be preferable. The most obvious difference is that elasticity has to be evaluated at a specific point, but that makes sense. Both will be limited by typical regression issues like ensuring you have a good model with no big flaws. Guy, I know that you’ve said that elasticity fails to take variance into account, but if there’s an equation to connect elasticity and coefficients, how true could that be? Perhaps there are other reasons to not prefer elasticity in this case, but I’m not familiar enough with it as a measure to say.

      • Guy says:

        Yes, coefficients and elasticity are fundamentally the same idea. They are different ways of identifying how much change you will see in Y for a given change in X. But neither one tells you how much variance there actually is in X. Let’s say I measured the role of race in determining partisan affiliation, and learned that being Black yielded a 90% likelihood of “Democrat” (compared to 35% for non-Blacks). That’s a very strong relationship, and will give me a big coefficient. Now, how much of the variance in party affiliation is explained by race? You have no idea — that depends on the racial makeup of the population. In Utah, it would explain virtually none of the variance in party affiliation (because everyone is white), but in Mississippi it would explain nearly all the variance.

        So knowing how “sensitive” WP is to changes in each variable is only half the story. To know how much WP is actually influenced by each one, you also have to know how much players differ in that statistic. Even an elasticity that looks “high” won’t matter much if the variance is low (players don’t differ much), while a stat with a “small” elasiticity — like assists at 1% — can have a big impact if the variance in that statistic is large (as it is for assists).

        • Alex says:

          If elasticity = (dy/dx) * (x/y), and the standardized equation is y = B*x (assuming x and y have been standardized), we should have elasticity = B*(x/y), where B is the standardized coefficient, which does take into account the variances involved. No good?

      • Guy says:

        No good. You’re switching to standardized values, which requires knowing the variance. Elasticity doesn’t “know” the variance. Let’s take PPS. A player takes about 16 FGA per 48, and scores about 1 point per FGA. So a 1% increase in PPS should increase WP by roughly .01*16*.033 = .00528, which as a percentage of WP (mean = .1) gives you .053. That’s almost exactly Berri’s reported elasticity. (Some of the other variables aren’t quite so straight-forward because position adjustments and team adjustments complicate things a bit, but basically the elasticities are just the WP coefficients adjusted for the mean of each variable.) Now, note that this calculation has no relationship at all to variance. The elasticity will be exactly the same whether the SD for PPS is .01, .05, or .25 (i.e. whether players differ a little or a lot in FG efficiency).

        You’re making this way more complicated than it needs to be. Elasticities don’t “know” anything about variance. So how can they tell us what proportion of WP is explained by any individual variable?

        • Alex says:

          From everything I’ve been able to find about it, the elasticity depends on the function you use to connect your dependent and independent variables; I haven’t found an example yet where it’s estimated directly from data as in your example (or Phil’s). Is there a reason I can’t use standardized variables to derive my elasticity function, other than you don’t want to?

      • Guy says:

        Alex: I’ve explained my view as well as I can. Per Dr. Berri, elasticity is the percentage change in Y resulting when you change X by 1%. Why don’t you see if you can derive his elasticities using standardized variables in your data set?

        But before spending time on that, I’d suggest you step back and look at the big picture. We already know that the standardized coefficients tell a completely different story than the elasticities in terms of relative importance of rebounds and PPS (unless you think Berri miscalculated his elasticities). So how can the two methods provide equally valid answers to this question? It’s not logically possible…..

        • Alex says:

          I assume Berri’s numbers come from his usual, full model. That isn’t one that I specifically remember checking out, and certainly not with all the adjustments at the player level. I also assume you would disagree with this?

          I would agree that they might give discordant results. I don’t think that means that the elasticities are clearly wrong; perhaps the standardized coefficients are wrong? I’m still waiting in hopes that someone will explain elasticities to me well enough that I can evaluate which is likely to be better here. Your view is very clear, and perhaps right (although it disagrees with the book in that link), but until someone can show me a proof that it’s impossible for elasticity to work in this situation, I’m not going to dismiss it.

  11. entityabyss says:

    I’m not here to argue or support what berri said. I’m pointing out that scoring (as a whole) has big effect in a player’s WP.

    Idk what your results for net fg points shows, but in andres’ post on the top tole players, it’s clearly seen that the top role players in regards to the scoring produced around the same amount of wins as the people in the defender role. Also, WP isn’t about simply rebounds, but rather net possessions (rebounds + steals – turnovers), so certain people (marcus camby, ben wallace) not only get credit for rebounding, but also not turning the ball over much more than they steal (actually, out of last year’s top rebounders, they’re the only 2 players who had more steals than turnovers).

    As for your argument on elasticities, that’s that. I’m talking about your argument that rebounds play the biggest role in WP. If this were true, then in andres’ post, you would’ve much bigger numbers for defender wins produced as opposed to scorer wins produced.

    Also, I made the argument that rebounding made net possessions go unnoticed. For example. Marcus camby, last year, was on one of the slowest (if not the slowest) paced teams in the league and averaged about 2 more possessions a game more than the next highest player (I believe is dwight howard). The reason for this is that he average .1 more steals than turnovers, when most high rebounders have a -2 in steals – turnovers. he got his teams +2 more possessions than the next highest person per 36 minutes on the slowest paced team.

  12. Guy says:

    “I’m talking about your argument that rebounds play the biggest role in WP. If this were true, then in andres’ post, you would’ve much bigger numbers for defender wins produced as opposed to scorer wins produced.”

    Not that it really matters, but that is exactly what Andre appears to show: he has 19 players with more than 5 defensive wins but only 7 players with more than 5 scoring wins, despite the fact he is combining FT and FG shooting while our discussion here — following Dr. Berri — was only about FG shooting.

    And if your points are all actually unrelated to the discussion here, as you seem to be saying, then perhaps you shouldn’t have begun by saying “There’s something wrong with this.”

  13. evanz says:

    “Your view is very clear, and perhaps right (although it disagrees with the book in that link), but until someone can show me a proof that it’s impossible for elasticity to work in this situation, I’m not going to dismiss it.”

    Alex, I think you’re misunderstanding the argument here. It’s not that elasticities “don’t work”. Of course, the math works the way it’s supposed to. It’s really more about inference from the regression, and what the model is telling you. The elasticities tell you how much the dependent variable changes according to a unit or percentage change in the dependent variable. Fine. The question you have to ask is, whether a 1% change in rebounding is “equal” to a 1% change in shooting (or whatever). In other words, is it *meaningful* to compare 1% change in these two variables? Don’t you see that the 1% is rather arbitrary? If the variance in one of the variables is much larger than the other, a 1% change in that variable may be easier to achieve. Therefore, we check the variance of the distribution as a better way to compare disparate “things” like rebounding % and shooting % (or whatever). Assuming each variable has a normal distribution, a change by one standard deviation represents something more meaningful.

    • Alex says:

      So we should not use elasticity when the variables involved have different variances? I’d be sort of surprised if that hadn’t come up in economics before, yet they seem to keep using it.

      • evanz says:

        Alex, I think it depends what you are trying to answer. If a “1%” change is meaningful, then use it. For example, if someone asked me, “How much do I need to change our rebounding to get X additional wins?” — in that case, elasticities have some meaning (of course, you could just directly use the regression equation, itself). But the question we have been wrestling with is, “Which one of these terms in the model should I focus on to get X additional wins?” To answer that question, it doesn’t make sense to simply use X%-change for each variable, because we don’t know if we could actually increase by X%…

        That’s really the core issue. If you increase rebounding by *some* amount, and you increase TS% by *some* amount, you can (within the limits of the model) calculate *any* arbitrary number of wins you’d like. But is it realistic? Can I say, hey, let’s increase our team’s TS by 10%! Yay, we’ll win a lot of games. Or, can we reduce our team’s TOV% from 13.1% to 5%? Sure, why not. We’ll win a lot of games! Of course, we can’t just arbitrarily choose the values like that. We have to have some idea of the distribution.

        In fact, another way of standardizing the data, could be to take the min and max values for each distribution, and look at the effects. It wouldn’t be any more “right” than any of the other techniques. It would just be a different way of asking the same question.

      • Guy says:

        I generally agree with what Evan says here. But when he says the question we have been wrestling with is, “Which one of these terms in the model should I focus on to get X additional wins?,” that’s not exactly right. We have been asking “which of these variables CURRENTLY plays the larger role in determining WP (and wins)?” Elasticities alone can’t possibly answer this specific question, because the answer depends on variance of the variables (as well as the elasticity). And when we look at the combined impact of variance and elasticity/coefficient, we find that rebounds explain more of the variance in WP48 than in actual wins.

        Practically speaking, though, my question and Evan’s likely have the same answer, since it will usually be easier to increase performance on a variable for which there is more variance.

        Alex, you keep trying to make this a debate over whether elasticities are legitimate metrics, when no one has ever said otherwise. I’m sure they have many legitimate uses in economics. They simply can’t answer THIS specific question, as has now been demonstrated to you repeatedly both theoretically and empirically.

        • Alex says:

          Evan, Guy, here are a few of the questions I’d like answered. 1) I think we all agree that NBA stats should be valued at the point and/or possession level, right? Neither standardized coefficients or elasticity does so; is one closer than the other? 2) Economists, I have to imagine, want to compare the importance of variables that have widely varying variances fairly often. Do they stop using elasticity then? A link or a reference would be appreciated. 3) Guy thinks one reason we should not use WP is because the team level and player level standardized coefficients don’t line up; the texts I’ve read say that we should be very wary about comparing standardized coefficients across different samples because they have different variances; elasticity doesn’t have this issue. Rebuttal? 4) I posed an equation relating elasticity and standardized coefficients. I was told it was wrong with no proof. Anyone have a proof? I know elasticity can’t be evaluated for variables with a mean of 0 (like standardized variables); does that mean it can’t be calculated at all, or just not calculated at the mean? If the equation is valid and we can relate the two, how could it be that elasticity “can’t” contain information about the variance?

    • Guy says:

      Well said. And to apply this to our specific case, a 1% change in PPS is a change of .117 SDs, while a 1% bump in rebounding is a change of only .047 SDs. So the same percentage change is far less common in PPS. And even if for some reason one doesn’t agree this means the increase in rebounding is “easier to achieve,” it is still a plain fact that there is more variance in rebounding. That higher variance is more than enough to offset PPS’s higher elasticity, and so rebounds do end up accounting for more of the variation in WP48.

      And I have to say that this — “until someone can show me a proof that it’s impossible for elasticity to work in this situation, I’m not going to dismiss it” — is quite a strange posture. Why should anyone bear the burden of proving it “impossible” for a particular tool to answer the question? One would normally require evidence that a tool IS the right one to answer our question.

  14. Pingback: Arturo's Silly Little Stats

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s