As promised last time, I have a few words for TMQ. In his most recent article he ‘decodes the secrets’ of the Packers, who are currently 12-0, already have a playoff spot locked up, and are the most recent team to be the focus of the ‘can they go undefeated?’ discussion. One of his points is that the Packers have a lot of undrafted players. TMQ has a long-standing love of undrafted players (or often traded players, or players let go in free agency – ‘unwanted players’) and contrasts them with first-round picks who are, virtually to a man, crybabies who are used to getting what they want and don’t listen to coaches. Teams with lots of undrafted players do well because those players do what the coaches ask and just want the team to win. Today we’ll find out if that’s true.
The data set: I downloaded the roster from pro-football-reference.com for each team from 2004 to the current year. I chose 2004 because that’s how far back my own data goes in case I want to line them up in the future. The Packers’ current roster, for example, is here. I grabbed the bottom full roster part, not the starters. That was for a couple reasons; one is that I don’t know how they define a starter (some teams have more than 11, and in previous years they included special teams, which I didn’t care about directly) and another is that their starters don’t always start a lot of games (right now Nick Collins is a ‘starter’ with 2 starts in 6 appearances; the Patriots have Danny Woodhead listed with 2 starts in 2 games). I also didn’t grab everything from that section; I wasn’t interested in college or birthday, for example. I did grab position, age, games started, years in the NFL, and draft position.
Once I had that for every team (and made a couple of corrections by hand, like assigning a draft position to supplemental choices and putting 22 for a couple guys with no age listed), I aggregated the data to the team level. The basic info for a team in a year is the average age, average experience (rookies are 0), average draft round choice (undrafted counts as 9), and average draft position (undrafted counts as 270). I chose 9 and 270 as dummies for anyone undrafted to reflect that they were particularly ‘unwanted’ – they’re far away from the 7 round, 215-ish picks that occur in the draft. If a team has a good number of undrafted players, these values should help shift their averages. I then defined a starter as any player with more than 7 starts and got the average age, experience, and draft position for each team’s starters. Perhaps there isn’t an influence unless the guys get on the field, after all. Then I counted up the number of rookies and undrafted players on a roster as well as how many qualified as starters, and finally I put in how many games the team won, their point differential for the season, and how many playoff points they got that year (0 points for missing the playoffs, 1 for losing in the first round, 2 for losing the second, 4 for losing the conference game, 7 for losing the Super Bowl, 11 for winning the Super Bowl).
For my analyses I took out the current year since the season isn’t over yet. But we can look at a few trends. For example, the leaders in undrafted players this year are the Browns and Rams, tied at 27. That isn’t so hot for the TMQ theory. But third is the Jets, and they aren’t all bad. But next are the Colts and Bucs. Hmmm. Maybe they have to start? In that case the leader this year is the Patriots with guards Brian Waters and Daniel Connolly, receiver Wes Welker, DBs Kyle Arrington and James Ihedigbo, and DT Kyle Love. Then we have the Bills and Jets, so undrafted starters run thick in the AFC East. Of course these are just impressions, so I’ll turn to the rest of the data (2004-2010) for more concrete answers.
Let’s start with something different, like age. My guess before looking at the data is that older teams will do better; they have veterans who are more likely to be quality players whereas younger players may or may not be helping the team. But maybe there’s a quadratic effect; if you get too old the physical tools are gone. Here’s the raw scatterplot of age and wins.
There’s actually no effect of age, either linear or quadratic, although both predicted effects are numerically there. If you use point differential there’s also no effect. If you only look at the age of starters, you get a significant linear effect with point differential but not wins, but it isn’t overly impressive. Your starters would need to be 2 years older on average to pick up an extra win but the range in the data set is less than 4 years. So somewhat surprisingly, it doesn’t seem that age has a big influence on winning.
Ok, so now on to the TMQ topics. Do teams with a lot of rookie players win more? No.
That’s the scatterplot for total number of rookie players on the roster by wins with the regression line plotted on top. You hear that teams that keep their draft picks do well, but I don’t see it here: more rookies on the roster usually means more losing. The data is sparser, but the effect still holds if you look at rookie starters. Teams that start more rookies (the scale tops out at 5) lose more games. The effect also shows up for point differential; every standard deviation of rookies on your roster (3.66) above average costs your team about 28 points.
How about undrafted players?
The correlation is negative, but it isn’t significant, and the effect is even weaker on point differential. So simply having undrafted players on your team is not going to help it, and there’s some chance that it hurts. But then, to my surprise:
Having more undrafted players start does help your team. Presumably if you’re starting an undrafted player you’ve realized that he’s pretty good, like a Wes Welker. In contrast if you’re starting rookies it could be high draft picks that you feel obligated to play even if they aren’t good (to get more experience, for example).
Speaking of experience, let’s look at that real quick.
There’s a bit of an effect for years of experience; it’s about 2 years for a win, which is significant. Starter experience doesn’t seem to matter as much, although the effect is there if you look at point differential. So generally you want guys who have been around a little while, which also fits with our negative effect of having rookies, although it isn’t a big effect.
Now to finish up let’s look at draft position more specifically, using round and pick number. Remember that undrafted players get entered as really low (i.e. round 9, pick 270).
Draft round has little effect on wins; nothing pops up with draft round, draft round for starters only, or looking at point differential. The same is true for the actual pick position. As you might hear on TV, players gotta make plays. It doesn’t matter what round they were taken in.
As a side note, I ran all the same regressions but predicting making the playoffs and playoff points, since some people mostly care about the playoffs. I mostly found the same results: draft position doesn’t matter, rookies hurt you, more experienced players helps you, and starting undrafted players helps you.
So there’s a bit of evidence on either side of TMQ’s argument. Having undrafted players on the roster doesn’t help on its own, so the 16 guys he listed in his article don’t matter per se. However, if you start some undrafted players, that does seem to help. To that end, the Packers have Tramon Williams, Ryan Grant, and Marshall Newhouse already over my starter threshold, and Tom Crabtree and Sam Shields could get there as well. Five undrafted starters would indeed but the Packers towards the top of the distribution. My guess isn’t that undrafted players work harder and listen to coaches more in general, but that if you happen across an undrafted player who is good enough to start you have a good player.
Perhaps just as importantly for the Packers, they only have 11 rookies and one rookie starter (the also undrafted Newhouse). So they have a couple signs of having a quality roster so far as it relates to winning this year. Overall the signs of a good team seem to be starting undrafted players (presumably because of selection for good players), and not having or starting rookies. Putting those three variables into a regression for wins, we can see which teams should be doing well this season. These predictions are obviously more limited in range than actual team outcomes (about 4 wins to 10, even though we’ve seen the Patriots’ undefeated season and the Lions’ unvictorious season), but they’re still interesting. By this measure the best roster of the past 8 years belongs to the 2009 Eagles, who started 7 undrafted players and only had 8 rookies on the roster (although 2 started). They won 11 games but lost in the first round of the playoffs. The worse team was the 2006 Packers, who started 5 of their 22 rookies but not a single undrafted player. They won 8 games, but had the point differential of a 5 or 6 win team. This year the top three rosters belong to the Patriots, Chiefs, and Steelers. Obviously the Chiefs are not living up to expectations. The bottom of the pile is the Browns, Jaguars, and Colts. Maybe this was a good year for Peyton to get injured?
Interestingly, the Packers are only in the middle of the NFL with respect to undrafted players this year. If TMQ were correct that the more the better, we would be looking at the Rams and Browns, as I mentioned earlier. And we should be wary of the the 49ers, Titans, and Bengals at the bottom of the list. Clearly, this year at least, TMQ should have known that simply having undrafted players is not an indicator of team quality. And while the Lions are indeed towards the top of the list this year for highest draft position (4th) and thus prime for crybaby-ness, it apparently isn’t bothering the number one team, the 49ers (or the Bengals for that matter). So I’m going to continue taking TMQ with a grain of salt, although he did manage to kind of have a point with undrafted players if you give him the benefit of the doubt and assume he meant something he didn’t say.
A different finding I’ll mention is that having rookies seems to be bad. This strikes me as maybe conflicting with the result that teams with more draft picks tend to win more. In that study, he summed team wins and draft picks over about 20 years and thus only had 32 points in the correlation. So there’s definitely a difference in methodology; I looked at how many rookies were on a team within single years. He notes that if you break up the data the results get much weaker; if I sum over teams in my data I only get a correlation of .07, so that matches up a bit (and it isn’t negative, as I found in the analysis above). So some of the difference is likely due to methodology. But I would also guess that some of the difference is again due to selection. Having a lot of picks doesn’t mean you have to have a lot of rookies. Instead you can look at a bunch of guys for free(ish) and decide which ones to keep. The Patriots, for example, lead the list in that article; they are drafting lots of guys. But they’re toward the bottom of the list in my sample, so they don’t have a lot of rookies on their team. So I think we can both be right; drafting a lot is good because you get to look at a lot of cheap labor, but having rookies is bad because for the most part they won’t be as good as guys who have stuck in the league.
Note: I realized when I finished that the scatterplots include points from the current season, but the regressions (and any regression lines) were run on only previous seasons.