Sounder Data: Is MLS losing its parity?

Greetings, compatriots!

This is shaping up to be quite the season, isn’t it? Teams around MLS are breaking records for both success and failure in 2018. Wednesday was the perfect example. We saw the Sounders, a team who now holds the record for consecutive regulation wins and consecutive playoff appearances, versus Orlando City SC, a team who now holds the record for worst-ever defense by goals allowed. Two teams are on pace to break the best points-per-game this year too: Atlanta United and New York Red Bulls are both averaging more than 2.03 points per game with two games remaining, which is the record since 2001. San Jose is way down at the other end with 0.625 points per game, good for fifth worst this century.

Remarkably, these records are being set more and more frequently. Orlando’s new title was claimed from Minnesota United, who set that record last year. Toronto FC set the modern record for points per game last year, too. It would appear that the gulf between the best and worst teams in the league is widening. It’s enough to make one wonder if MLS is still the “parity league” it was once made out to be. It’s enough to make one spend an entire evening analyzing data to find out ...

Welcome back to Sounder Data, the really-infrequent series where I use data, statistics, and visualizations to answer deep questions about the Sounders and MLS!

This time we’re looking at the concept of parity, and trying to assess whether it’s declining in MLS.

To the unaware, parity in sports is the idea that teams are more or less on a level playing field. That even though some teams are better than others, the competition is close and games and seasons are usually exciting while dynasties rarely if ever are established. If you want to learn all about it, fellow Sounder at Heart sidereal wrote a superb three-part series (part one, two, three) on the concept of parity back in 2010. It’s really worth a read.

To paraphrase, there are three ways to think about parity:

1. Franchise parity: whether the same teams are good year after year (or bad year after year)
2. Season parity: the gap between best and worst teams in any given year (or the variance)
3. Game parity: the chance that the underdog will win in any given game

For today’s purposes (and because I can’t for the life of me figure out how to measure #3), we’ll just be looking at the first two types of parity.

1. Franchise Parity (tl;dr the gap is widening)

First up, franchise parity. If “good teams” are staying the “good teams” more consistently, then we should see it in the data.

As the above-mentioned series points out, franchise parity really just comes down to the correlation between last season’s results and this season’s results, year after year. Here’s a picture of that for every season since 2001 (one year after the rules changed):

The diagonal lines show the correlation, whether the previous season’s points are similar to the present season’s points team-by-team. To put it another way, each dot represents a team. If the dot is in the upper-right corner, that means the team was really good two years in a row (the year labeled and the previous one). Lower-left means a team was really bad two years in a row.

The point is that franchise parity means uncorrelated seasons, i.e. flat lines. You can see that some years, like 2003-2005 had really strong parity (remember, parity means level playing field), but other years like 2011-2013 and 2015 had really weak parity. The same teams that were good before were good again and vice versa.

Returning to the question, if parity in MLS is on the decline, we would expect to see these blue lines getting more positive over time.

As luck would have it, there’s a statistical test for that. I ran a quick regression to measure a thing called an interaction term, which tests whether the slope of that line is gradually increasing. If it is, it would imply that franchise parity is gradually decreasing. If you like equations, here’s an equation:

If that looks like nonsense to you, it’s fine. The point is that it helps us measure whether franchise parity is decreasing.

It turns out, it is! On average, the slope of the blue lines above is getting steeper each year by a small margin. That margin is statistically significant (if that’s what you’re into), with a p-value of 0.026. Here’s what it looks like when you estimate the coefficients in the equation and stack them all on top of each other:

As we get closer to today (the lines and dots get bluer), the correlation between last year and this year’s results gets stronger. Parity is getting worse.

2. Season Parity (tl;dr the gap is widening, but not by much)

Next is the idea of season parity. If the gap between the best and worst teams is getting bigger, or even if the teams are just generally getting more spread out, we should again be able to see it in the data.

This is neat because the spread-out-ed-ness or clumped-togetherness of data is something us statisticians spend sleepless nights pondering. So much so that people have come up with dozens of ways to measure it. For this application, I think there are five good ways to look at spread: range (distance from the worst team to the best), inner-90th percentile (distance from the 5th percentile to the 95th), inter-quartile range (distance from the 25th percentile to the 75th), standard deviation (basically the average distance from the average team), and median absolute deviation (basically the median distance from the median team). So, what are those statistics doing over time?

For each of these things, a high spread means weak parity. For example, the largest-ever gap between the best and worst team (the green line) in terms of points at the end of the season was in 2005, where the best team (San Jose with 64 points) finished 46 points ahead of the worst team (Chivas USA with 18 points). That’s outrageous for a season with only 12 teams.

However, 2018 might break it. At the time of writing, The gap between Atlanta (66 points) and San Jose (20 points) is also 46, and we have two games remaining. There have been a couple other years recently that came close too, like 2012 and 2013, which both ended with a range of 43. 2010 and 2011 were also pretty high, especially on other metrics, and so was 2017. It’s worth noting that seasons are two games longer than they used to be though, so there is technically more room for a gap to develop.

But is the spread, and therefore season parity, gradually getting wider as time progresses? If you squint your eyes at the graph above, you can kind of see a general upward trend in each line. Or, I can make a graph for you that makes it clearer:

It’s funny how that works isn’t it? A moment ago you could sorta-kinda see the upward trajectory, and now it looks clear as day. That’s a sign that the correlation isn’t super strong and I’m trying to spin a narrative.

In fact, it’s not super strong evidence. While every measure of spread is going up each year on average, the trend isn’t statistically significant for any of them. The strongest is the range, with a p-value of 0.06.

Still, it’s been an unusual season for parity, a lot like 2005 was. We’re tied for biggest gap between the best and worst, and just shy of the biggest gap (39 versus 42) between the best 5 percent of teams and worst 5 percent of teams (that inner 90th percentile metric I showed). 2018 has also been fairly high for the other three metrics, ranking 5th, 4th and 4th for them respectively in the last 18 years. It’s certainly noteworthy.

---

So where does this all leave us? Well, there are many ways to slice the data. But I’ll say that by my eye, parity is getting weaker in MLS. It’s especially apparent if you look at the standings year-over-year, but you can also see a widening gap between the best and worst teams within each year. As time goes on, I’ll be eager to see whether the trend continues.

It’s curious, however, to debate what the ideal is. Do we want perfect parity where every game is a coin flip? Surely not. Do we want more parity than we have today? I don’t know.

As I am wont to do, I’ll leave you with some things to ponder about the limitations of the data I’ve just presented to you. First, there are some statistical issues. I’m analyzing temporally auto-correlated data without adjusting for it, and I’m using linear regression when assumptions of continuous normal data may not apply. You’re right to question whether the above p-values are perfect frequentist statistics, because they are not.

More importantly though, these data do not tell us why this is happening. I refer you back to sidereal’s three-part series (part one, two, three) for some really thoughtful discussion on the facilitators and barriers to franchise parity and season parity. One driver is probably expansion. As the league gets bigger, the meaning of parity changes, and the quantities I’m measuring above change with it. Maybe parity isn’t getting worse, the league is just getting bigger. It’s hard to say.

Another driver of parity though is the league’s salary structure. This is certainly an important issue today, as TAM, GAM and DP slots seem to be growing on trees. In fact, in the face of rumored cut-backs on DPs slated for 2020, I wonder if MLS already knows parity is widening and they’re trying to do something about it. I also wonder if it’ll work, and whether we, supporters of a team who has come out on the top, even want it to work. I’ll leave it to you all to speculate.