Sounder Data: How important is lineup consistency?

tl;dr: Lots of changes to a starting lineup are bad.

Hello again, Sounders at Heart!

This week the Seattle Sounders lost 2-1 to Portland in the US Open Cup. Last week they lost 2-1 at Montreal. That was preceded by a 2-1 loss at Dallas, a 3-2 loss at Kansas City and a 0-0 tie at Philadelphia. Oof. That’s a tie and four losses in a row for you mathy people. The optimist in me will say that the play on the field hasn’t been all-out pathetic, but this stretch has not been fun as far as results go.

Granted, none of these games were played at CenturyLink field, a place that offers a statistically-great home field advantage (but you avid readers already knew that). And granted, the Sounders played some pretty good teams in this stretch, both in the standings and in terms of current form. But we’re a more-than-pretty-good team! What gives?

There’s been a fair amount of discussion on the pages of this lovely website that the reason is absences. Sounder at Heart senior editor Jeremiah Oshan pontificated about it earlier this week, and a well-spoken yet inexplicably self-deprecating fanposter hashed out how Seattle’s absences compare to LAFC a little before that. They both made excellent arguments that fielding a less-than-first-choice lineup is obviously bad, but that the lack of consistency from week to week is also bad, and that alone is a big reason why we keep losing.

And it left me wondering: is it the inconsistent lineup itself that has hurt us, or are our backups just bad?

It’s not an easy question to answer, especially the second part. But what service would I be to you if I didn’t at least try? I should probably try.

Welcome back to Sounder Data the series that’s becoming less and less frequent where I use data to answer tough questions about the Sounders and MLS. This time: lineup consistency.

In the loss to Montreal last week, the Sounders fielded this starting lineup. In the game before it against Dallas, they fielded this very different lineup. All told, there were six players in the Montreal lineup who weren’t in the Dallas lineup at all (Nouhou, Saad Abdul-Salaam, Jonathan Campbell, Will Bruin, Víctor Rodríguez, and Handwalla Bwana). It wasn’t so bad against FC Dallas, but the Sounders still had to field a lineup with three changes from the week before. At Kansas City? Five differences. At Philly? Another five differences. Week after week, that’s a lot of changes.

Especially when you compare that to our successful run to the start of the season. Look at the numbers so far this year and you can see where things started slipping:

Okay, so the first game of a season always has a lot of new faces in the lineup, so that’s excusable. But the next five games included only one change among them! As you can see (and surely remember), the results were great during that stretch. The next five games included only a single win however, and lo and behold, lots of lineup changes. Two, then three, then four, then three, then three during those games. Seattle only had a single change on May 16th — a game which they won — then started the latest stretch, which I just described was both bad and had a huge number of changes.

This is looking like a trend to me. Just from the Sounders’ games in 2019 alone, it looks like when we have a similar lineup to the previous week, we’re more likely to win. It’s not cut and dry — there are some examples where Seattle won despite lots of lineup changes and lost despite few lineup changes — but there’s a notable pattern emerging.

So I pulled up the numbers for the rest of the league. Better yet, I pulled up the numbers for every game dating back to 2015. With all the data together, the trend is pretty visible:

The first bar here represents all the games MLS teams have played since 2015 in which they fielded a lineup that was identical to their previous game, irrespective of who was playing or when. The win percentage? Exactly 50 percent of 170 games (shown in the dark part of the bar). Every bar after that shows the games played with one fewer player in common from the previous game that team played. When you pool together all the games where the team fielded an identical lineup aside from one player? 45 percent win percentage (of 477 games). Two differences from the previous game? 38 percent win percentage.

Looking across the last four-plus years of MLS games, a team’s win percentage declines with every additional change to its lineup. The notable slight exception is between three and four changes, where it goes from 36.3 percent to 37.6 percent, but even that is a decline if you include the draws (the green parts of the bars). There’s also an exception between seven and eight, but I’d chalk that up to small numbers. By the time a team is suffering with nine changes from their previous lineup though (something that has only happened 22 times since 2015 mind you), the win percentage is a measly 18 percent.

What’s that? Did you ask if this is just chance?

Finally, a question about whether something is just chance. No, no it’s not. I ran a quick statistical test (a logistic regression with “win” as the outcome), and found that for every additional new face in the lineup compared to the game before, a team’s probability of winning declines by an average of 2.5 percent. As an odds ratio, that’s 89.9 percent lower odds of winning for each extra personnel change. With a 95 percent confidence interval of 88.2 percent to 91.7 percent, that’s statistically significant AF; the p-value is 8.91e-08.

Furthermore, there’s nothing I can think of that would otherwise explain this pattern. It’s not like teams who have fewer available players are more likely to play good teams, and it’s the quality mismatch that explains the trend rather than the lineup consistency. For the most part, lineup turnover is the consequence of the randomness of injuries (something that affects everyone) and seasonal international call-ups (something that is independent of who teams happen match up against during that time). We can basically think of it like a natural experiment. Lineup changes just happen, and the fact that they’re correlated with win percentages isn’t reverse causation.

One could argue that coaches choose to change their lineup when they’re afraid of an upcoming powerhouse, and that could explain what we’re seeing here. If true, that would explain it, but I suspect that’s limited at best. You could make a more convincing case that playing on the road (which is bad for your chances of winning) comes with a higher chance of lineup turnover. That would explain what we’re seeing here too, but the difference is only slight. Away teams average 2.88 lineup changes per game, and home teams average 2.63. I even ran the aforementioned logistic regression controlling for home field advantage and got effectively no difference in the results.

So (in part), this points to the same thing that I’ve been pointing to a lot lately: staying healthy matters a lot. Teams that avoid injuries are more likely to win. Of course, that’s not going to make overlapping international and MLS games go away (although there are people in this world who could literally make that go away). When those two things happen, expect lower odds. And while the MLS schedule remains the way it is, I guess we just have to hope we’re playing opponents who also have lots of national team players, or at least hold on to our seats when we aren’t.

At the very least, this does lend some solid evidence to an idea that people have been kicking around. We’ve dropped a lot of points lately, and a lot of it can be explained by turnover. Let’s get those guys back from international duty, eh?