Parity: What Is It And How Much Is Right? (Part 2)

In Part 1 last week I described some of the issues that come up in defining and measuring parity, I broke parity down into three distinct types (franchise, season, and game) and I came up with one possible calculation for franchise parity. In this part I'm going to look at season parity.

Season Parity

Just as franchise parity is a consequence of the roster construction rules of the league (including salary caps, drafts, etc), season parity is a consequence of the seasonal competition rules of the league — particularly the playoff rules. How many teams make the playoffs? How are they organized into subdivisions? Are wild cards involved?

In a league with high season parity, the end of the season is a tense period with a large number of teams close to the cutoff for making or falling out of the postseason. The 2009 MLS season was considered to have a large amount of parity in part because the playoff run went down to the last week of the season, whereas the 2010 season was considered by many to have relatively less parity because the playoff teams were decided with some weeks to spare.

Methodology

The methodology for season parity was, by far, the most problematic for me to define. While franchise parity is a very straightforward correlation between results in consecutive seasons and game parity is (as we'll see next week) similarly straightforward, season parity doesn't lend itself to an obvious formula. It would need to capture the individual effects of divisional standings and wild cards in creating playoff chances, so it can't simply be an examination of winning percentage or points percentage league-wide.

Originally I added the end-of-season games behind data to a spreadsheet and added a 'games ahead' for teams that made the playoffs (on the argument that being way into the playoffs is as bad for parity as being way out of the playoffs). But it became problematic when I tried to decide whether to use games behind or winning percentage behind. Games would favor leagues with fewer total games — in essence I'd be saying that being 1 game behind in the NFL (out of 16 total) is equivalent to being 1 game behind in MLB (out of 162 total). This would make the NFL seem much higher parity since teams are rarely more than 5 or 6 games out (and almost never more than 10). On the other hand, if I use percentages then we have the opposite problem. A team 1 game out in NFL is 0.0625 back, while a team 10 games back in MLB is 0.0617 back, which would suggest that the NFL team is further away from a playoff place, which is nonsense. There are some normalizing tricks I could try to help bridge the gap (for example, I invented a stat called the WOG — the Week Of Games), but combined with the fact that we were only measuring the last day of the season, the whole approach was dissatisfying.

Instead I decided to go to the simulation well. Regulars on the site know that I have a Monte Carlo simulation setup for the MLS season that I use to track playoff and championship odds throughout the season. It's similar to Sports Club Stats, but with a few improvements. It takes into account all of the playoff rules of the league, including wild cards, so you get an accurate representation of a team's likelihood of making the playoffs no matter its division.

So I could use it to calculate the playoff odds of each team before each game in the season, creating a population of 480 (16 teams times 30 matches) playoff percentages. Then I could examine the distribution of that population. In a league with low season parity, the odds would quickly collapse to either 100 or 0 and a large number of samples would be at those edges. In a league with high season parity, the odds would instead cluster around PlayoffTeams / TotalTeams — around 50% for MLS and around 27% for MLB, for example. So if we simply take the proportion of samples that fall within some range we call competitive, we can determine how competitive the overall league was over the course of the season. But creating these simulations is a lot of work, and though I plan on getting around to leagues other than MLS, it's way beyond the scope of this project. Fortunately, Ken Roberts at Sports Club Stats has already done it, and I like the results well enough that I'm comfortable using them. Note that I'm using the 50/50 model (which assumes that every team has as 50/50 chance of winning future games) rather than the weighted model (which biases future results based on past performance) because I want to isolate the effects of the divisional structure rather than the relative strengths of the teams, which is going to be covered in the article on game parity.

The standards for competitiveness are somewhat arbitrary, but I decided that any sample between 0.75 and 0.25 was competitive, meaning that any day your team has between a 25% and 75% shot of making the playoffs is one in which your team is still competing for a playoff spot. This range could be increased or reduced within reason and it would change the absolute parity numbers (increasing them as you widened the range), but wouldn't affect the relative rankings of the leagues.

So the plan is to collect three seasons worth of playoff percentage data, then calculate the proportion of results that are within that range. Straightforward enough.

The EPL Problem

Unfortunately, the EPL presents a problem. While all of the US-based leagues use a playoff system to determine which teams can compete for the championship, the EPL, like most soccer leagues throughout the world, declares a league winner without an elimination bracket. There are really three ways to translate this into our methodology.

The least generous to the EPL (in terms of parity scores) is to translate 'playoff odds' to 'championship odds'. Since the EPL doesn't have a playoff, then your team is only 'competitive' if it has a shot at the top of the table, which is definitely rare. More generous would be to acknowledge the importance of the European place, which acts as a significant incentive to teams that aren't likely to win the league. This expands the number of competitive teams by expanding the number of target positions from one to anywhere from 4 to 7 (depending on whether you count the Europa League). Even more generous would be to acknowledge the importance of the relegation zone by including the odds of not being relegated as part of the measure of competitiveness. If we define season parity by the amount of drama that's brought to the league at the end of the season, then the relegation zone seems relevant.

Unfortunately, the availability of data constrains our choices. Sports Club Stats only measures championship odds and relegation odds, and only championship odds are made available in the format I'm using. And I think this is the right choice anyway. In the end, parity is about competitiveness and the prize every team is competing for is the championship (either via the playoffs or not). So the EPL will have to live with championship odds and the resulting hit to our calculated season parity

Results

A good way to visualize the data that leads to these results is to take a look at the playoff/championship odds graphs at Sports Club Stats. For example, here's last season's EPL data (the graph of interest is the one in the top right corner of the page) and here's the 2008 MLS data which was the highest single season parity in the group. You can see the number of teams that stay in contention in the MLS versus the very early collapse of the EPL race into just a few teams.

The EPL result really is strikingly bad. It means that only 5% of the time did a team go into a match without being very confident of either having no chance to win the title or having very likely won it. By comparison, in the MLS 54% of pre-match fan bases were unsure about their playoff situation. I think that should be a significant caution to anyone who wants to uncritically eliminate the playoffs from MLS play in order to more closely match European leagues.

And speaking of MLS, here are the specific results for the seasons I looked at:

Although you have to be careful about drawing too many conclusions from a small sample as you start looking at single seasons, there's been a pretty dramatic reduction in season parity over the last three years. This fits with the general understanding that this year's playoff race was less close than last season's (and apparently the previous season's, though I cared about that one a lot less without the Sounders). If this isn't just noise, it might be partly a result of the proportion of playoff teams. There's been 8 playoff teams through all 3 seasons, though the number of teams has grown from 14 to 16. Next season we'll see the results of having 10 playoff teams (out of 18 total).

On the other hand, the MLB and NBA results argue against the impact of increasing the number of playoff teams. MLB has the smallest proportion — not counting the EPL — of teams making the playoffs (at 8 of 30, or 27%) while the NBA has the largest (at 16 of 30, or 53%). And yet they have essentially equivalent season parity results. To some extent this is because increasing the number of playoff teams yields diminishing returns as far as late season drama goes. For every team in the bottom half that's given an increased chance to make the playoffs, there's a team in the top half that's even more guaranteed of a playoff place — a problem that plagues the NBA where teams are often perceived to be coasting out the season with as much as half the season left. One option to increase season parity without this problem is to keep the number of teams low but increase the number of divisions. The NFL maintains high season parity in part with 4-team divisions, in which a team only has to beat out 3 competitors to earn a playoff spot. Of course, the NFL also benefits from an extremely short season, which is something that no other league will want to emulate.

From an MLS perspective, these results are pretty complimentary toward the current setup. Rather than last week's franchise parity results, which suggested that the MLS is bafflingly random, these more closely match with the NFL results, which are always a good target to shoot for. But this doesn't suggest that some of the more bizarre elements of the current setup (like this season's Eastern Conference Final) need to be maintained. Instead the league could maintain season parity by going to a larger number of smaller divisions with guaranteed playoff places and fewer wild cards. Regardless, this is definitely the category of parity among the three that's going to undergo the most flux over the next few seasons.