UPDATE! I added some new content below at about 6 on Monday, if you happened to read it before then.
Despite only winning by a touchdown, the Lions looked really good this weekend against the Bucs, a young team (like themselves) who won 10 games but just missed the playoffs last year. Matt Stafford threw for 305 yards, but more importantly had a high efficiency (9.2 yards per dropback) while only throwing one interception and not getting sacked. The running game wasn’t great, but it wasn’t terrible and there were no turnovers. On defense the Lions held the Bucs to about the same mediocre running production but a much lower passing efficiency, only 5.6 yards per pass attempt to go along with one interception and two sacks. The game was only so close because Stafford’s interception was returned for a touchdown and the Bucs got their first points after a big kick return. What can Lions fans expect going forward?
Toward the end of last year, I took a look at how teams change their ranking over the course of the season. The main impression you get is that they become more similar as the season goes on; they start somewhat spread out and then move closer. The main reason for this is probably regression to the mean; teams that do really well or really poorly early in the season were playing over (or under) their heads early on and can’t sustain that level of performance beyond maybe 6 games. Brian Burke noticed that post and mentioned that for this very reason he adjusts performance for each stat in his model. I thought I would look at my data to see exactly how much you can tell about how a team will end up at different points in the season.
My NFL data covers from 2004 through the end of last season (I always have to hold off on getting new data until after the Monday night games, so I sit at my desk all antsy during the day on Mondays). For each team I grabbed some of the key stats used by my model (and Brian’s): passing efficiency, run efficiency, interception rate, and so on. Then for each team I matched up their stat after week 17 (which reflects their performance over the course of the whole season) and the same stat during each earlier week of the season. For example, last year the Lions ended up with a pass efficiency of 5.77; they averaged 5.77 yards for every dropback. But in week one, they had an efficiency of only 4.11. In week two, their efficiency for the game was 7, raising their ‘season-long’ efficiency to 5.75. Each of those numbers (4.11, 5.77, etc) gets matched up to the 5.77 and so on for every other team. Then I can pull out the week one values and see how week one pass efficiency predicts season-long pass efficiency, and so on for the other stats.
As you might have guessed, week one tells you very little about what a team will be like at the end of the year. The equation I got for passing efficiency is 5.06+.165*week one efficiency; with the Lions getting 9.2 yards per pass yesterday, we would predict that over the course of the whole season they’ll have a pass efficiency of about 6.58 yards per dropback. Thus they’re due for a pretty strong regression. But we can’t be too sure because the R squared is only .178; they could regress more or less. How about running? The story is about the same; the R squared is only .15 but if we had to predict we would say the Lions will have a run efficiency of about 4 yards per carry. On the other hand, week one interception rate has virtually nothing to do with a team’s interception rate at the end of the year; the R squared is .04. So Pittsburgh fans can rest easy that they won’t see three picks every week and the 11 teams that didn’t throw any picks shouldn’t expect that to hold up. Penalty rate was actually the most consistent stat I looked at, having an R squared of .22 starting in week 1.
So when do you know what kind of team you have? It depends on what you want to consider ‘reliable’. Say you want an R squared of at least .7 to feel like you have a good sense of how the team will look at the end of the year. Pass efficiency gets there after week 7 when the R squared hits .73; if the Lions are still somehow getting 9.2 yards per dropback after week 7, we can expect that their season-long efficiency will be 8.33; there should still be some regression, but not nearly as much and it will be more reliable. Run efficiency, however, doesn’t get there until week 11. In fact, penalty rate and completion percentage hit that threshold in week 8 but everything else waits until week 11. Thus you have to get pretty far into the season before a team’s stats look similar to how they will at the end of the season.
That result is perhaps somewhat intuitive, since the week 17 number is an average of all the games that came before it. Thus the week 11 number, for example, has about half of the information that will go into the week 17 number. On the other hand, it might be surprising that it takes that long for a team to ‘look like’ what their actual quality is. Sports writers will start crowning Super Bowl favorites and teams looking forward to the draft after another game or two (they already have, to some extent), but we really have no idea what these teams will be like at the end of the season. Last year Seattle looked pretty decent after week 7, standing at 4-2 (they had a bye) with wins over Chicago and San Diego and with 4 more games against their crappy division as well as the Raiders ahead. Then they went 3-7 the rest of the way. Something of a counter-example is the Eagles, who were a similar 4-3 after week 7. That’s when Vick returned healthy and went 6-3 the rest of the way, including a last-minute loss to the Cowboys where Vick and other starters were rested. The Eagles illustrate another source of noise in the predictions, which is injuries or general changes in personnel. It’s just hard to say what’s going to happen in the NFL.
So in short, Lions fans shouldn’t be overly optimistic just yet. Week one turned out better than a loss, of course, and it’s better to play well instead of poorly, of course, but week one just doesn’t say much about how teams will look at the end of the year. We’ll have a much better sense of how good the Lions are after week 11 or so, which is good because after that the Lions still have two games against the Packers and one each against the Saints and Chargers. Hopefully at that point the model says the Lions are pretty good, because otherwise any playoff hopes might be sunk by a tough end to the schedule.
UPDATE – So while playing around with the numbers some more, I tried fitting a model predicting week 17 from the other weeks but without an intercept term. That rocketed the predictive power right up; week 1 passing efficiency predicts end-of-season efficiency with an R squared of .909. I’m not sure why that is, but the increase happened for all the stats, although occasionally to a lesser extent: interception rate is only at .489 in week 1 but hits .9 by week 6 and fumble rate starts at .68 and gets to .9 by week 4. This fits much better with Brian’s work, since he stops adjusting the numbers after week 4, I believe. Obviously these predictions would be more bullish on teams that did well even after one game, at least as far as passing and running efficiency go. Turnovers are still hard to pin down.