Your Guides for the Season

I thought it was high time to give an updated description of the model I use for predicting NFL games, as well as describe the new one I’m going to test-run this season, so here it is!  This post will mostly be a reference and will take over as the NFL link in the header at the top of the page.

For the curious, here are a few pages describing previous iterations of the model and things that will be relevant shortly: first, roughly describing the models I was starting with; second, when I changed from my ‘every predictor possible’ model to Luigi, a slimmed-down version; third, a post that looks at some of the variables that predict winning games in the future (obviously applicable to a model that seeks to do that very thing); and fourth, a look at how my team ratings change over the course of a season.

Ok, so with that background knowledge floating around, let’s talk about the models.  My primary model is Luigi.  Luigi isn’t actually a single model but three models, one to predict the probability of the home team winning a game, one to predict the total number of points scored in that game, and a third to predict how many points the home team will win (or lose) by.  The common thread across these three models is the predictors that go in to them, and that set of predictors is what makes Luigi Luigi.  My earlier model, which was again really three models, used many more predictors and was called Mario.  Sadly, Mario didn’t adapt well and so has fallen by the wayside.  So let’s focus on Luigi.

Luigi is an inherently predictive model.  It’s a regression model that takes certain statistics for the two teams playing a game and predicts who will win.  However, it only takes statistics available before the game happens.  For example, if I’m trying to predict if the 49ers will beat the Lions in week 2, I only use statistics for the 49ers and Lions from week 1.  This is because I want to try and avoid overfitting.  Fumbles and special teams play can greatly affect the outcome of a game, but that doesn’t mean they are reliably going to occur again in the future.  By specifically predicting forward in time, I hope to reduce the impact of those less reliable statistics while still including those that increase predictive accuracy.  This isn’t perfect, as can be seen by the example of Mario.  When you include everything, even less reliable statistics will influence how the whole prediction turns out, and can make it worse than need be.

The regressions for total points and point differential are typical linear regressions that most readers of this blog should (I think) be familiar with.  The regression for win probability is slightly different in that it’s a logistic regression.  Instead of predicting a number that’s fairly normally distributed, like total points scored in a game, I’m predicting a ‘yes’ or ‘no’ outcome, in this case ‘win’ or ‘lose’.  Once you fit the regression, it makes predictions in terms of probability.  Also, all regressions are framed towards the home team, so ‘probability’ is probability of the home team winning and point differential is positive if the home team is expected to win and negative if the away team is expected to win.  So for a particular game Luigi might say the home team has a 60% chance of winning.  Obviously they have to either win or lose (I guess they have a tiny chance of tying), but the claim is that if teams with the statistics of the home and away teams were to play each other a bunch of times, the home team would win 60% of the time.

So that’s Luigi, and I like him alright.  This year I’ll also have two other sets of predictions (which will surely make for messy blog posts, but I’ll figure it out).  Due to possible issues of overfitting with a typical regression, I thought I would check out another kind of regression.  Namely, I went with a regularized regression (here‘s the wikipedia article, and here‘s some work I did with regularization).  One of the features of regularization is that it picks the regression weights by what makes the best out-of-sample predictions, which is obviously something we’d like to have here since we’re trying to predict what happens in games that haven’t happened yet.

Sticking with the theme, my regularized regression predictions will be called Yoshi.  But that isn’t easy enough; I’m going to have Yoshi 1 and Yoshi 2.  Yoshi 2 is exactly the same thing as Luigi but with a regularized regression; same data set, same predictors, etc.  Yoshi 1 is a little different in that I wanted to give the regularization a chance to tell me if it had to do more ‘work’ early in the season as opposed to later in the season.  For example, you tend to see more extreme numbers early in the season because fewer games have occurred and so team averages can be very noisy.  Later in the season the averages have settled down for the most part.  So it’s possible that more regularization is needed early in the season while less is needed later (and Yoshi would be relatively similar to Luigi).  Also, there’s some chance other fundamental things change across weeks in the NFL.

The big difference is that Yoshi 1 only uses games from the same week as the game being predicted; if I’m trying to figure out who will win a game in week 2, it only uses the results of previous seasons’ week 2 games (Luigi and Yoshi 2 use every game, assuming that week doesn’t matter too much).  This cuts the sample down a lot, but at this point I still have over 100 games for each week, so hopefully it isn’t an issue.

To summarize: Luigi uses a typical regression and makes a guess based on every other game in the database; Yoshi 2 is Luigi but with a regularized regression; Yoshi 1 only uses games from the same week as the game being predicted and can thus have a different amount of regularization (as well as different weights on each predictor) for each week.

A couple suggestions of things to look for: Luigi has been terrible at over/under picks, so it doesn’t do very well predicting total points.  Maybe one of the Yoshis will pick up the slack.  Regularization generally has the effect of moving weights closer to 0, and thus moving predictions closer to the mean.  So look for the Yoshis to produce predictions generally in the same direction as Luigi but not as extreme.  And look for a good year of football!

This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

8 Responses to Your Guides for the Season

  1. Squad says:

    Can’t wait to see the yoshi’s in action. I’m really curious as to the results from yoshi 1. Using games from the same week from years prior strikes me as odd but nonetheless has increased my curiosity. May I ask, how many years back does the yoshi 1 data go?

    • Alex says:

      Well, the idea is that whatever predicts winning in week 2 might be different from what predicts winning in week 11, for example, even if it’s just how much weight to give the predictors. Like, maybe in week 2 turnovers are just too noisy and need to be cranked down a lot. So Yoshi 1 will only use previous week 2s to predict this week 2, and only previous week 11 to predict the week 11 a couple months from now, and so on. It could also be completely unnecessary; we’ll find out!

      My data goes back to the 2004 season, so there are 8 complete seasons in there and we’ve just started the 9th.

      > Date: Wed, 12 Sep 2012 23:57:12 +0000 > To: >

  2. mike says:

    Loving the site… here’s a link to a bunch of data, best $30 ever spent!

  3. Eric says:

    How do you address multicolliearity? Ever try partial least squares regression or discriminant analysis?

    • Alex says:

      I’ve tried using PCA on the predictors before and then using the PCA factors as predictors, but I don’t recall it making much of a difference. The two Yoshi models use regularized regression, which reduces some collinearity issues. I haven’t looked at the correlation between my predictors in a little while; I’m positive it’s there, but I don’t know how much there actually is at this point.

      • Eric says:

        Right. At the end of the day, it doesn’t really matter unless you are trying to estimate the betas. Though in terms of improving accuracy, I’ve found that PLS discriminant analysis yields noticeably higher accuracy (e.g., looking at ROC curve) than PCA regression. So may be worth checking that out. Here’s the R package I use:

        This is Eric from grad school by the way. Hope all is well!

        • Alex says:

          Thanks Eric! I totally didn’t pay attention to the email address the first time. Yeah, I don’t even bother looking at the betas since I just want the best prediction (also why I haven’t really bothered trying to cut down on variables at all), so I don’t sweat any collinearity issues for the most part. But better accuracy is always nice.

          Things are going pretty well. On the job market now, which kind of sucks. Hope you and Stefania are (still?) enjoying Jersey!

          > Date: Mon, 16 Sep 2013 17:38:07 +0000 > To: >

          • Eric says:

            We are enjoying Jersey, thanks. Good luck with jobs. Unfortunately, I still remember doing that a couple years ago. Godspeed!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s