## A Few Replies

My last post generated a few comments and they seemed like they deserved a little more room than my own comments in reply (plus I made some graphs!), so here we go.

First, the issue of using percentages as a measure.  We were specifically talking about rebound percentage, which basketball-reference.com calculates as the percentage of available rebounds a player gets (you can also do it separately for offensive and defensive, which would be out of own-team misses and other-team misses, respectively).  In general, I like rebound percentage.  It naturally accounts for things that differ across players and teams, like pace, teammate shooting ability, time on the court, etc.  Reggie Evans, Dwight Howard, and Zach Randolph all get 12.1 boards per game, for example, yet Randolph plays a minute more per game than Dwight and 9 minutes more than Evans; on the other hand, Memphis plays much slower than Toronto or Orlando.  But by looking at rebound percentage, I can see that Evans is a better rebounder because he gets 26.3% of missed shots while he’s on the floor while Howard gets 20.7% and Randolph gets 19.3%.  Presumably if Evans played more, or if Randolph’s team played faster, they would get more rebounds.  But at any pace or playing time, Evans is apparently the best rebounder of the three.

However, rebound percentage has the disadvantage of being a percentage.  That means it must be between 0 and 1.  When we look at how a percentage varies with some other variable, the function is often somewhat S shaped; it starts low and climbs quickly, becomes more linear in the middle, and then slows down as probability approaches 1.  A number of functions can be used to describe the shape, like the logistic or probit.  As an example, here’s a graph with two lines, one made using a logistic-like function.  The line to the left has the same ‘slope’ as the line to the right, but a different intercept.  This changes where y=.5 hits on the x axis.

Below is a graph with two more functions.  Both have intercepts of 0, so y=.5 is at x=0, but much smaller slopes than the functions above.  One is a more gradual increase than the two above, which I would say is more typical of functions like these, and the other has a slope so small that it looks linear in the range I plotted.  It would only approach 0 and 1 at much smaller and bigger values of x, but rest assured that it would flatten out when it got there.

The main point I want to make here is that in every case, functions flatten out as they approach y=1, as they must since probabilities cannot be greater than 1.  Their main difference is in the slope, which describes how quickly they move from low to high probability.  The lines in the first graph have high slopes and so there is a quick jump from 0 to 1; the slopes in the second are smaller and so the jump is more gradual.

So now we can turn to diminishing returns.  Since we are using probabilities, there will ALWAYS be diminishing returns regardless of if we are talking about rebounds, shots, or anything else.  The only question is at what point on the x axis it starts; perhaps the slope should be the measure of ‘how much’ diminishing returns a statistic exhibits (although I’d still be wary of measuring it this way).  The leveling-out towards 1 starts extremely early in the first graph, a bit later for the S-shaped line in the second, and will start very late for the nearly-straight line.

This is why I said that I wouldn’t necessarily use rebound percentage to look for diminishing returns because a) I would always find it if I looked at the full function and b) the amount of diminishing returns I find would depend on what portion of the x axis my data fell in.  In the data presented at countthebasket, for example, projected offensive rebound percentage is at a lower range than projected defensive rebound percentage.  Eli concludes that there are greater diminishing returns for defensive rebounds, but they could in fact be described by the same equation; defensive rebounds simply are higher on the x axis and so the actual probabilities are becoming smushed while actual offensive rebound probability is in a more linear section of the curve.  If we were looking at a league where steals happened all the time, we would be talking about diminishing returns in steal percentage.

The S-shape of probabilities not only guarantees diminishing returns, but at the low end of the x axis it makes the interesting prediction of “accelerating returns”.  That is, adding N points of whatever x is at the low end will move your probability up more than adding N points at the middle or high end of x.  I don’t have the actual data to fit, but I think you might get this for the steal and block data here.

So if I were going to look for diminishing returns in the same way that the two linked articles above did, I would not use rebound probability.  Since the projected and actual values come from a player’s season value compared to line-up level value from play-by-play data, there’s no need to account for pace and the like (unless certain line-ups within a team play drastically differently even when the same player is in both line-ups).  As I have said before and commenter Guy suggested, rebounds per 36 minutes should be fine.  Also I would not add up probabilities regardless of the statistic being investigated.  Probabilities do not add linearly (in the second link, you actually have a projected assist rate greater than 1, which is impossible).

Regarding the weighting of variables and how they relate to summary statistics, it might not be the weight per se so much as the relative weighting.  I said that my fake WP48 was more consistent because it put a higher weight (.3) on fake rebounding than fake NBA Efficiency (.25) or fake PER (.15), and fake rebounding was the most consistent measure.  But it’s also true that fake WP48 has a higher relative weighting of fake rebounding to fake shooting efficiency (which is less consistent); the ratio is .3/.1 = 3 for fake WP48, 1 for fake NBA Efficiency, and .3 for fake PER.  I’d have to play with more numbers to see if the absolute weight or the relative weight is important.  In either case, I recommend reading Ty’s post on the Blazers for a counterargument to the importance of rebounding in WP48.

Finally, some words on consistency.  In the way I was using it last post, consistent is the same thing as predictive, because I was looking at consistency across years.  If something is consistent, I can be confident about what it will be like next year, which means I can make predictions.  Jason Kidd rebounds well this year so he will probably rebound well next year; he is shooting poorly this year but he might shoot better (or worse) next year.  Consistency is to be valued, but it shouldn’t be imposed (by using some algorithm that maximizes a correlation) and it doesn’t have to occur.  For example, I have never seen research that finds that much of anything in football is consistent.  Some statistics do correlate (at least within a year), but usually weakly; I would say that this means they are only somewhat consistent.  Less consistent measures lead to noisier predictions, which is why picking 55% against the spread is amazing.  All the statistics, as well as our eyes, tell us that the Patriots are one of the best teams this year, yet because of small sample sizes and inconsistency it is hard to be confident about how well they will play next game (or next year).

On the other hand, some things do appear to be consistent.  Year-to-year rebounding, blocked shots, and assists (per minute) at the player level all appear to be pretty stable (correlations of .87, .87, and .9 according to Stumbling on Wins).  There isn’t really a way to know why this is; I could speculate that it has to do with a combination of a player’s ability and having a stable role on a team even if he switches teams (big men will continue to be asked to rebound and defend the basket; point guards will continue to move the ball).  Similarly I can only speculate as to why shooting percentage is less consistent, and I have absolutely no idea why free throw percentage only has a .59 correlation across seasons.  All we can do is measure these things and then use them the best we can.  My claim in the last post was simply that if you combine a number of measures, it appears that your summary measure will be consistent to the extent that it heavily weights (or relatively weights) consistent measures.

This entry was posted in Uncategorized. Bookmark the permalink.

### 16 Responses to A Few Replies

1. some dude says:

Why not just take everyone rebound % and adjust it to the league average pace?

If Dwight Howard rebounds 20% of shots and Orlando plays slower than league average, why not just adjust the RP48 numbers up to the pace (meaning, if Orlando played faster and everything else is equal, what would his RP48 be?).

I still don’t agree with consistency. I can just as well argue the Sun orbits the Earth because I consistently see the sun set in the West and rise in the East every day. I can predict that it will happen with 100% accuracy. Doesn’t mean my conclusion is correct or that there is much value in that consistency. If I try to measure stuff based on a flawed assumption that the Sun revolved around the Earth, my overall predictions will fail.

And that’s the heart of the issue. We see that individual rebounding, and thusly WP48, is consistent over time, but we don’t see the fruits of that knowledge help much in predicting the future, which seems odd since something so consistent over time doesn’t make predictions easier at the team level. Again, you’re touting consistency for consistency’s sake alone, not whether there’s value in that consistency. It’s like saying you’d rather be consistently wrong over time than wrong some of the time and accidentally right some of the time. Um, okay…how about we just try to figure out how to be knowingly right all the time?

• Alex says:

You could adjust rebound per 48 for pace and I think that would be a good step, but someone would still complain that Bogut gets more chances to rebound because Milwaukee shoots poorly and Garnett doesn’t get enough because Boston shoots well (or similar examples). Rebound % takes care of that. My main point is different tools for different tasks. Rebound % will often be great, but once in a while it won’t be.

As I said in my other post, consistency is valuable all else being equal. If a model is obviously irrevocably flawed, then you’re right that it doesn’t matter much if it is consistent. But in my opinion, WP is least flawed out of a variety of options, and (separately) it has the benefit of being as or more consistent than the others. I know that other people disagree, and so it would be right to dismiss WP’s predictive value from their viewpoint.

• some dude says:

I don’t think you understood my point on RP48.

I’m saying to take a players rebounding % and adjust the actual figures to pace. For example.

Say Dwight Howard is exactly 20% Reb rate. Say the league average pace is 100 possession and Orlando plays at 95. Now let’s say Dwight got 1000 rebounds and since we know his rate, that means there are 5000 chances when he played. Now we adjust THAT number to the pace. How many rebounds more would be occurring at a pace of 100? Divide 5000 by 95 and get about 52.63 and multiply that by 100 and get 5263 rebounds at a 100 pace. 20% of this is about 1053 rebounds. Now divide that by 82 games and minutes and multiple by 48 and you got your pace & % adjusted RB48 numbers.

Now do this for every player in the NBA and you will get your figures combining Reb % and adjusted for pace and the issues of boundaries AND “more shots available” are eliminated. I’m not adjusting RB48 only to pace, but also to rebound %.

back to consistency, again we’re assuming players are supposed to be consistent over time, but I’m not sure if that’s accurate. WP seems to be best at explaining the past in terms of the team level, but is less predictive at the individual level in predicting team wins. That’s a major flaw and I believe it has to do with valuing consistency for consistency’s sake.

• Alex says:

No, I took your point. I was just saying that instead of going through all the algebra, I could stay with rebound % to compare players. If I needed to know how many rebounds exactly to expect from someone I would go through the math, but if all I want to know is who gets more boards, rebound % is fine.

Can you tell me where the wins produced method makes a choice based on consistency? My understanding of it is that it starts with scoring efficiency (offensive and defensive) explaining wins, then points and possessions explaining scoring efficiency, which leads to the rough estimate of a point = possession. Then box score stats are valued according to how they affect points or possessions, hence offensive and defensive rebounds being equal along with one point scored and steals, and turnovers being worth the same amount but negatively weighted. I don’t see where consistency comes in.

2. Guy says:

Alex: Thanks for the thoughtful reply. On the weighting of variables, relative weighting is the only thing that matters. Indeed, weighing has no meaning except in relative terms — the weight assigned rebounds in a metric is only “large” or “small” compared to the weights of other variables. For example, one reason the WP overweights rebounds at the player level (apart from diminishing returns) is that it underweights scoring. A player who shoots 2-for-4 in 48 minutes gets the same win score as one who goes 15-for-30. So that greatly constrains the effective weight assigned to scoring, which then makes rebounds account for a large share of WP. And keep in mind that what we’re calling the “weight” of a factor within a metric reflects two things: the coefficient in the metric, and also the variance of the statistic among players. (Variance for player Reb48 is very large.)

On the “percentage problem,” I would make two basic points. First, your argument does not show that DR doesn’t exist at high and low percentages, only that they are inevitable at some extreme values. They are still quite real. So, for example, if it turns out to be physically impossible for an NBA team to have a dreb% above .77 (or whatever), then that would be a finding of very severe diminishing returns on drebs! So at most, your argument says that Witus’ findings were at some level inevitable, not wrong. (Although I don’t think this is true.)

Second, you are misunderstanding the impact of boundaries on oreb% and dreb%. You worry that the finding that dreb% has larger DR than oreb% is just a function of a boundary. But that’s impossible, since they represent equally extreme values — the average oreb% of .23 is just as close to zero as dreb% is to 100% (which mathematically must be true). Suppose Witus had used dreb%(opponent) instead of oreb%? Now our mean is .73 rather than .27, but the results will of course be exactly the same around the new mean (since oppDReb% is just 1 – OReb%). So we know for sure that the much larger DR on defensive boards has nothing to do with boundaries.

Look closely at Witus’ dreb% graph. If your theory were correct, DR should be much larger above .73, as the “boundary” is approached and DR become inevitable, but smaller under .73. But we don’t see that at all — in fact, DR are slightly larger at expected 60% than expected 85%. Also, your solution of just looking at raw rebound totals, while effectively controlling for pace and opportunities, is just a disguised way of using reb%. Think about it: if you control for pace and missed shots, then reb48 will have a perfect correlation with reb%! So you’re just pretending not to use reb% to feel better, and either method will tell you the same thing.

If I still haven’t convinced you, you could just do his analysis again using total reb%. Since that averages 50%, you would minimize the risk of a boundary problem. What you will find is large diminishing returns overall, not quite as high as for dreb%. So that will reassure you his findings are not a function of approaching a boundary. However, it is valuable then to look at oreb and dreb separately, since the DR effect appears to be very different.

Finally, on consistency. Consistency is great, but only IF you are measuring the right thing. Consider this: rebounding and height are very highly correlated. So let’s create a metric that divides credit for rebounds on a team proportional to height. This metric will correlate extremely well from year-to-year, probably the same .9 we see for actual rebounds. So players will deliver the same every year — hooray! But I can’t imagine you would find this a good metric (and the heads of some WOW bloggers would explode when Bargnani was credited with a 7-footers’ share of rebounds!). So let’s not elevate consistency over actual measurement of what creates wins and losses.

• Alex says:

Guy – re: consistency, you can see my comment to some dude. In regards to weights, I’m not aware on any work that includes the variability of a statistic in its weight. Can you point me toward some?

In terms of diminishing returns, I’ve said twice that I think they occur. My points have simply been about the best way to measure it. I fundamentally disagree with the simple adding together of probabilities, which is what has been done so far to get projected rebound rates. Your point about the symmetry of offensive and defensive rebounding is correct and well-taken, but it assumes that the actual function is symmetric. I haven’t seen anyone fit anything other than a linear function, so who knows if this is true. I don’t have the data to do this, but if you would like to repeat Eli’s analysis using projected rebounds per minute compared to actual rebounds per minute, I would be much happier with the description of diminishing returns from those graphs. There would be no issue with adding together rebounds per minute and no boundary issues in any direction. You could fit the data with a linear regression to demonstrate that the actual data fall off below a straight line as projected rebounds increase, and then fit an exponential or power or some other function to describe how much drop-off can be expected.

• Guy says:

“Your point about the symmetry of offensive and defensive rebounding is correct and well-taken, but it assumes that the actual function is symmetric. I haven’t seen anyone fit anything other than a linear function, so who knows if this is true.”

Sorry, but I have no idea what this means. Who is assuming that what is symmetric?

“I don’t have the data to do this, but if you would like to repeat Eli’s analysis using projected rebounds per minute compared to actual rebounds per minute, I would be much happier with the description of diminishing returns from those graphs.”

I don’t see the point. I’ve explained how the boundary problem you were worried about is not an issue. Furthermore, there is no escaping the fact that rebound opportunities are finite. You can use pace-adjusted rebounds and pretend you’ve escaped that constraint, but you haven’t.

Remember also that you have a much easier way to figure this out, using a simulation. As we discussed, you can just compute team dreb48 variance assuming player productivity is independent — that’s the zero-DR scenario. Then compare that to actual team variance in the NBA. The difference between the two will give you a good measure of diminishing returns.

And let’s keep things in perspective: at least for drebs, your concerns about using reb% amount at most to a quibble: Witus showed that dreb% is virtually insensitive to the players in a lineup, and this was just as true below 73% as above. So you already know that diminishing returns are massive, and therefore that WP is massively misvaluing high-rebound (and low-rebound) players. Indeed, the level of DR is so large that it would literally be more accurate to simply treat every player as a position-average rebounder than to credit players with a full rebound. I’m not advocating that as a good metric, of course, but it tells you how inaccurately WP is handling rebounds.

3. Guy says:

Alex: Still wondering if you have any thoughts on how it can be right for WP48 to weight rebounds very heavily (as you indicate it does), when rebounds and wins in the NBA are not strongly correlated. A player’s WP48 is determined much more by his rebounds than his shooting efficiency, but real wins correlate strongly with efficiency and only weakly with rebounds. Do you have a theory that would reconcile these two findings? How can rebounds simultaneously account for most of the “wins” produced by players, yet account for relatively few actual wins?

• Alex says:

Man, you’re quick. Well, in that Lewin and Rosenbaum article that never got published, rebounds per minute has a .65 correlation with wins produced at the player level. According to Arturo’s post, total rebounds at the team level has a correlation of about .35 with wins. I imagine the difference has to do with transitioning from rebounds per minute to absolute rebounds (e.g., not all players play equal minutes, so their contributions don’t equally sum up to the team level). Also, looking at simple correlations (as opposed to partial correlations or similar measures) loses some meaning in a multivariate environment. I could similarly ask why PER correlates with shots taken at .668 but shots taken correlates with winning at only .34, or why Rosenbaum’s Alternate Win Score with his preferred rebounding weights has a correlation of only .08 with turnovers yet turnovers have a -.435 correlation with winning. I don’t have good answers for any of them except that presumably something changes when you move from player per-minute measures to the team level, and that the variables in question are explaining different things (a player measure versus team wins).

• Guy says:

“I imagine the difference has to do with transitioning from rebounds per minute to absolute rebounds (e.g., not all players play equal minutes, so their contributions don’t equally sum up to the team level).”

Sorry, Alex, but this just doesn’t compute. The difference between .65 and .35 is enormous, and we’re using Reb/minute here, not reb%, as you prefer. If you say that 1) WP48 correctly distributes productivity among players (more or less), 2) WP48 is driven largely by rebounds, and 3) the sum of player WP48 equals team wins, then it must follow mathematically that winning and losing teams will differ greatly in their rebounding. But that is not what we see — right now, the 10 worst teams are outrebounding the best 10 teams. And typically, the best teams are only slightly above-average in rebounding. So we know for sure that rebounding is not a big part of success in the NBA. If a player metric is driven largely by rebounds, then it cannot possibly be correctly measuring players’ contributions to winning. It really is that simple.

• Alex says:

I think we have a few points of disconnect. First, WP48 is not largely driven by rebounds, it just emphasizes the value of rebounds relative to other stats more than some other metrics. I could point out some counter-examples if you’d like; the top five guards in WP48 right now are Chris Paul, Dwayne Wade, Deron Williams, Rajon Rondo, and Russell Westbrook (I took out Landry Fields and Ginobili because the automated site has them partially as forwards; basketball-reference has Fields as a forward). These same guards are 27th, 6th, 73rd, 45th, and 19th in guard rebounds per 36. Paul is the number one WP48 player in the league and second in wins produced. It seems like it would be difficult for this to be true if rebounding was the primary driver of their WP48.

Secondly, as I said before, we are working in a multivariate world here. Simple correlations are at best hints at what’s important and at worst completely misleading. I assume you have the Wages of Wins book given the level of your interest in it; I’d point you to page 98, which has a discussion of simple correlations and what they might tell you in basketball. For example, offensive rebounds are negatively correlated with wins. Perhaps we should develop a measure where players who get offensive rebounds lose value? None of the statistics occur in a vacuum, and so simple correlations are not an important discussion point.

• Guy says:

“WP48 is not largely driven by rebounds, it just emphasizes the value of rebounds relative to other stats more than some other metrics.”

I suppose this depends on what you mean by “largely driven,” but we can say more precisedly that 1) rebounds are by far the most important determining factor in WP48, and 2) rebounds have far more impact than shooting efficiency (even though we know that the reverse is true in terms of producing actual wins in the NBA). I’m sure you would agree that anecdotes are not a good way to answer this question. In a multivariate model, which you correctly say is the right approach, rebounds is by far the largest determinant of a player’s WP48. By that I mean that rebounds account for more of the variance than any other factor. Most importantly, rebounds account for far more of the variance than does shooting efficiency. Using Arturo’s 2009-2010 data, I find that a 1 SD change in Reb48 yields a .069 gain in WP48, while a 1 SD gain in Pts/FGA yields just .o35. So rebounds have about twice as much impact on a player’s WP48 as does his shooting efficiency. (And all other factors are relatively small, except assists for PGs.) Arturo’s data file is available at his site, so feel free to do your own analysis.

So your caution about simple correlations is correct in principle, but in this case the simple correlations tell the same story as more precise multivariate estimates. At the player level, rebounds drive WP48 far more than anything else. If you knew only every player’s Reb48 and I knew only their TS%, and we both tried to predict players’ WP48, your predictions would be vastly better than mine. However, repeat that exercise at the team level and I will win every time: the high-efficiency teams win far more than the high-rebound teams.

So again, don’t you think this suggests major problems in WP? If we know that wins are produced mainly by shooting efficiency (and reducing opponent efficiency), rather than rebounds, shouldn’t a good player metric reflect that reality?

• Guy says:

And your example of orebs is very helpful. The only way orebs and wins could be negatively correlated — when getting orebs is obviously a plus — is if differences in shooting efficiency dwarf differences in offensive rebounding ability. And that’s the reality we see: teams vary much more in their ability to put the ball in the basket than in their ability to secure rebounds.
But when WP48 apportions credit among players, it tells us the opposite (and incorrect) story.

4. some dude says:

Alex, I wasn’t talking about comparing players, I was talking about your simulation where you want to ignore reb%. I’m saying to simply adjust the rebounds to pace based on percentage and then run it. This fixes the “boundary” problem you claim in the post.

I never said WP makes any choices regarding consistency. I’m saying it overvalues a stat that is consistent over time because it assumed that the consistency is valuable, when we in fact don’t know if that’s true and DR suggest it may not be.

• Alex says:

Where does it assume that consistency is valuable?

• some dude says:

Sorry, the formula doesn’t, the people who believe it doesn’t need to be tweaked think it’s valuable.

That’s what I meant.