Is Running All Better Now?

In Brian’s recent post on run success at Advanced NFL Stats, he shows that running is more important to winning than he thought previously.  Running efficiency (yards per attempt) doesn’t correlate well with winning in the future; instead passing seems to be king.  However, if you use success rate instead, running does correlate (although still not as much as passing).  One of Brian’s conclusions is that coaches must be optimizing something like success rate, which is a play-by-play measure, instead of big-picture goals like drive success or game success (which we know coaches don’t maximize because they run too often and go for fourth down conversions too infrequently).  I think that’s close to true, but not quite.  I think that coaches are optimizing for first downs, which has the side effect of making runs more successful, and successful plays by definition will tend to lead to winning.

We should start with a couple notes on running and passing to start.  I got Brian’s play by play data for 2008 and 2009 and found the average, median, and mode values for run and pass plays.  The data isn’t perfectly parsed, so I think interceptions probably affect the pass data, but on the whole I think the numbers should be fairly accurate.  In the past two seasons, the average run has gone for about 4 yards, while the median and mode are 3 and 2.  For those of you unfamiliar with the terms, median means there are equal numbers of longer and shorter runs while mode means the most common value.  So the next time you’re talking about how you have to run to win, keep in mind that the most likely thing to happen on a run is that you gain a big two yards.  On the other hand, the average, median, and mode for pass plays are 6.7, 5, and 0.  So passes tend to go for more yards, although the mode tells you that the most likely thing is that the pass will be incomplete (not compared to a completion, just compared to any other particular number of yards gained; league-wide incompletion rate was around 34%).

The run numbers are what I’d like you to focus on here.  How is it possible that teams can win, or even have a successful play, if the mode run is only 2 yards?  ‘Success’ is defined by Brian as improving your expected points; he also provides the rules of thumb that coaches are more likely going by.  For example, you want to get half of the yards for a first down on first down; on third down you want the yards necessary to convert for the first down.  The expected points definition is derived from Brian’s model and tells you the average points scored by a team with a certain down and distance to go at a certain point on the field.  I got EP values here to compare first and ten to second and six at various points on the field.

Using the EP values just mentioned, I looked at what you would need to get on first down for the play to be ‘successful’.  For example, if you had first and ten on your own 30 (70 yards to go), you would be expected to score .9 points.  If you then ran for four yards making it second and six on your own 34 (66 yards to go), you would have an EP of .87 points.  So your run for four yards has cost you .03 points, which isn’t much but certainly isn’t an improvement.  Looking at the range over the whole field, a four yard run on first down is only a success starting around midfield.  A three yard gain on first down is never successful.

So how can running be successful when a three yard gain (or less), which happens half the time, is fairly useless on first down? By getting to third and two if possible.  On third and two you can run the ball and be fairly certain of picking up the first down (since a two yard gain is the most frequent outcome, and teams get two yards or more about 68% of the time).  In contrast, a pass will get two yards or more only 58% of the time.  The EP for third and two following a first and ten from eight yards away (such as moving from first down at the 30 to third down at the 38) completely dominates being at second and seven; it’s better than second and six if you’re inside the opponent’s ten yard line; and it’s actually better than having first and ten if you’re inside your opponent’s thirty yard line.

So I don’t have direct proof, but I think that the relatively high success rate for running comes from when coaches can get to third and short, where running becomes more worthwhile and actually outperforms passing.  It’s kind of amazing that the success rate gets as high as it does given that coaches run more often than they pass on first down anyway.  Coaches are more likely to run than pass with any distance to go on first down, but are only more likely to run than pass on second down if there are 5 or fewer yards to go and actually only run more often than pass on third down if it’s third and one.  I think that shows a few things.  Coaches are overly conservative on first down, running far more than they should (which Brian has also shown), coaches try to set up a short third down on second down by running on 2nd and 5 or less, and that coaches are somehow overly aggressive on third down and pass far more often, even on third and short, when they could run and more consistently get the yards they need for a first down.  This sudden over-reliance on passing could be behind Brian’s finding that running is advantageous on third down, but not first or second.

To recap: because running doesn’t pick up many yards, it is almost impossible for running to be successful unless the run is called in short-yardage situations.  However, coaches run more often than pass on first down, on second and short, and on third and one.  I think this indicates how coaches are overly conservative and are trying to set up a short third down that they can expect to pick up easily, although it’s odd that once they get there they suddenly turn pass-wacky.  This would imply that coaches are optimizing towards picking up first downs, and the play-by-play success is more a function of that goal as opposed to specifically trying to ‘win’ each play individually.  In either case, it’s certainly true that they could likely do better in terms of optimizing overall expected points or wins by thinking about the whole game instead of this particular set of plays.

A final note: it isn’t surprising that runs becomes correlated with wins if they are measured in terms of success.  A successful play, by definition, is one that has improved your expected points.  More points makes you more likely to win.  So any time a team can be more successful, by running or passing, they should win more.  We might still expect run success to be less connected to winning than pass success because success glosses over the benefits beyond success (e.g. a 2 yard gain on 3rd and 1 is just as successful as a 20 yard gain by this measure), and we know that passing is still more likely to gain more yards in general.  So a successful pass should be ‘more successful’ than a successful run, and thus be more likely to lead to more points and more wins.