#Disrespekt: Delving into the ESPN's FPI Metric

Earlier this week, the Big Ten Blog at ESPN released their preseason FPI projections for the East and West divisions. In the East projections is an interesting nugget for MSU:

Projected overall record: 7.79 wins, 4.32 losses

Projected conference record: 4.72 wins, 3.28 wins

Odds of winning the Big Ten: 5.9 percent

If you happened to watch college football last year (or really, even if you didn't) these numbers might surprise you. The Spartans are coming off of a 13-win season, including wins on neutral fields against Ohio State and Stanford. The postseason FPI ranking for MSU was 28th. What gives?

The Football Power Index (FPI) Explained

One of the reasons these projections bothered me is that the man behind them is Dean Oliver. Mr. Oliver is the father of four-factor basketball analysis, which is kind of a big deal. This is not some crackpot system.

The FPI, at a high level, attempts to take margin of victory, split it into offensive, defensive and special teams components, and then adjust these categories for level of competition. For 2013, FPI says that against a perfectly average team on a neutral field, MSU would win by 17 points. The MSU offense would contribution 3.74 points, the defense would contribute 12.05 points, and special teams would contribute 1.21 points. Unfortunately, I don't know how they do this splitting of categories, because they don't give these details anywhere on the ESPN site.

Margin of victory is an attractive way to come up with team ratings because it captures everything that happens on the field. Because of this, Oliver suggests, "Among all the metrics ESPN is putting out, these are the most inarguable ones." Well, I'm not going to argue that the general idea is a good one. But what I am going to argue is that the outcome is biased.

Quantifying #Disrespekt

After wading through the FPI data, the distribution of efficiencies on offense and defense stood out:

via s1.postimg.org

via s23.postimg.org

According to FPI, there were no defenses which would produce more than 15 points of value against an average team on a neutral field. However, there were 13 such offenses. FPI is saying the best defense in the country (FSU at 14.98) is not as valuable as the thirteenth best offense in the country (Missouri at 15.27).

This points to an old question: what wins championships, offense or defense? FPI makes a bold claim in this regard. Of the FPI top ten, only two teams had an offensive efficiency below 15: Washington at 14.88 and Stanford at 13.61. By contrast, six of the top ten have defensive efficiencies below 10, including Ohio State at just 1.05.

FPI also (probably) doesn't see the offensive improvement by MSU over the course of the season. The Western Michigan and South Florida debacles are still data points in this analysis.

It's easy to see why FPI yields such a poor projection for the Spartans, then. Take an undervalued defense and factor in that only 5 starters return. Take a mediocre offense (over the course of the season) and factor in mild improvement because 7 starters return. Voila, you have a 8-4 team.

Haters Gon' Hate

The other thing I wanted to address is the way the Big Ten Blog handled these projections. The writer of the piece, Josh Moyer, has a spectacular lack of understanding of these ratings. Here are his comments:

"For one reason or another, the Spartans just aren’t getting much love here. The Wolverines are actually projected to win more games. Overall, Michigan State is favored in nine contests but is only a 55-percent favorite or better in six of those. ESPN Stats & Info plans to update these numbers once the season starts, so it wouldn’t be much of a stretch to think these odds will end up changing in the Spartans’ favor. But, for now, the odds just don’t like MSU."

The numbers will change over the course of the season, sure. But that doesn't account for how the FPI systematically undervalues defense. The numbers ESPN spits out aren't just "the odds." They come from underlying assumptions and the implementation of a system. When results don't make sense, you don't just throw your hands up in the air and move on. These comments are lazy, and show why someone with understanding of the projections system should be writing these projection posts.

It's hard to prove journalistic bias in sports coverage, but it's easy to prove systematic bias in data models. Here are MSU's FPI rankings from 2010 to 2013: 40(!!!), 19, 50, 28. In that span, MSU collected two Big Ten titles, two Legends division titles, three bowl wins (including the Rose), and a 3-1 record against Michigan. I'll be taking these projections with a grain of salt.