I took last week's newsletter off because I was enjoying my brief vacation between summer internship and school, but we're back now, and with a lot to catch up on.
The rundown
(in no particular order)
- 1.
Sporting Kansas City posted no one, not two, but three data-related jobs last week. The club are hiring a data scientist, data engineer, and data analyst, which would immediately rank them among the best teams in MLS in terms of analytics head count. The postings come after Francisco Belo, Nottingham Forest's former head of football intelligence, was hired as SKC's VP of Data and Analytics. The job postings are promising and seem to ask for all the right things1. As an example, the data analyst posting includes pre-/post-match reporting, benchmarking, recruitment support, and research projects among the role responsibilities.
- 2.
New England Revolution also posted for a Soccer Data Analyst position that offers a salary range of $58,000-$68,000 per year and similarly includes responsibilities surrounding player recruitment, opposition analysis, and more. The description is good but the pay is not.
- 3.
A few days ago, Kieran Doyle — who has been on a tear recently — posted an article to the ASA website analyzing how MLS Next Pro data can be used to predict a player's future performance in MLS. The sample size included a few hundred league transitions, where players played sufficient minutes in both divisions, and contained a ton of interesting stats for each player, including g+ categories. Kieran came up with a few key takeaways, including which metrics are more and less predictive of future output. The first finding was that passing is both dependent on teammates and independent of them; Kieran found that progressive passes are mostly player-determined, whereas passing g+ — value accrued from playing all kinds of passes — is more dependent on teammates making good runs and getting into dangerous positions. Dribbling, meanwhile, is almost entirely player-determined, which makes sense. Kieran found that ball carrying transferred the best between leagues, and my guess is this probably had to do with the relatively small number of variables impacting a 1v1. While 1v1s are not the only way to accrue dribbling value, it's an interesting type of game moment because it has a relatively low number of confounding variables. Rather than having to worry about the reaction times of five different defenders and the receiving ability of the player you're passing to, 1v1s are almost entirely determined by the attacker's dribbling skill and the defender's 1v1 defending skill. It makes sense that true outlier skill in this category would translate between leagues well. While this ideal, isolated situation is fairly infrequent, and most of a player's dribbling value comes from other areas, I think it's an interesting example that highlights how dribbling is generally more independent of external variables than other game situations. Overall, Kieran was able to reduce error in translation predictions by 42% for outfielders, which is a strong result for such a low sample. Translation models have always been of interest to me, and I'm really excited to see something like this out in the public.
- 4.
On Bluesky, Matt Doyle pointed out just how cracked Cavan Sullivan is at 16:
- 5.
- 6.
This is not normal for a teenager, let alone one of his age. The kid is averaging 0.28 xA per 96 minutes and 0.22 xG per 96, as well as high progressive passing, progressive carrying, and dribbling g+. In all comps, Sullivan now has 5 goals and 11 assists across 1400 minutes, which puts Sullivan among the best prospects to ever emerge in MLS. Compare Cavan's radar to that of Alphonso Davies in 2017, and the difference is stark. There are levels to this, and Cavan is breaking records.
- 7.
Despite being one of the worst teams in the Championship last season, one of the most fraudulent promotion sides of recent history, and a shoo-in wooden spoon candidate, Hull City managed to make beating Manchester United look easy over the weekend, which crushed both me and my soul. Manchester United should probably fold and end my suffering. No, but, for real, Hull City had a dismal set piece record last season, with a -0.16 set piece xG difference per 90 minutes. They were one of the worst teams in the season in terms of set piece xG. And they scored two set pieces against United. This is a great case study on how set pieces are really really important, and how being actively bad at set pieces can really hurt you. United were bad at set pieces in this game, and even though Hull were also bad, they were taller and quicker to the ball, and that was the difference. Maguire cannot mark a box all on his own. United need to fix this, quick.
- 8.
Austin FC announced last week that former Minnesota United CSO, Khaled El-Ahmad, would be taking over the CSO role in Austin effective immediately. KEA had been in charge of Minnesota's sporting structure for two and half years, and oversaw the hiring, as well as departure, of Eric Ramsay as head coach. General feelings from Minnesota fans are that KEA struggled to meaningfully improve the squad, with James Rodriguez's signing, and subsequent eight game stint, hardly helping that perception. KEA will work with Jim Curtin in Austin when Curtin takes over as head coach in the offseason, and the pair have a lot of work ahead of them. Austin rank second to last in the Western Conference, and despite splashing the cash in recent seasons for MLS-proven DPs, have struggled to find goals. In Minnesota, Head of Recruitment Hank Stebbins will step in as interim Sporting Director, with Stebbins seen as a candidate for the full-time position. Stebbins is well-regarded within analytics circles, and has led Minnesota's data operations for a few years. Stebbins also comes from a legal background, and worked as an assistant coach at the collegiate level. Like El-Ahmed, Stebbins will have his work cut out for him, and I'm excited to see what he does in the his new position.
A closer look: RAPM
RAPM is a controversial subject in soccer. Analytics transplants from other sports yearn for it. Soccer fans themselves can't help but be attracted to it. Even those of us who know its limitations want, in some way, for it to be possible. The jaded analyst will tell you, however, that it is not possible. Until recently, I myself fell into this camp.
Regularized Adjusted Plus-Minus (RAPM) is an advanced version of the simple plus-minus model, which aims to describe how well a player's team performed while they were on the field. Did they perform above their usual average (plus)? Did they perform below it (minus)? Across enough samples where a player is and isn't playing for their team, we can begin to gauge their importance to that team. As you can imagine, though, there's a lot of noise in there. If we apply some fancy statistical techniques, and use more granular metrics as the response variable, like xG instead of goals, we get more robust results.
Detractors argue that RAPM cannot truly succeed in soccer because of the naturally low number of substitutions in the game, which makes addressing collinearity in the model difficult. Proponents say that with enough data, the collinearity issue becomes less pertinent. Many attempts have been made at constructing such a model that is built to accommodate this challenge and in the spirit of exploration, I want to take some time to review these models rather than dismissing them out of hand.
The truth is that plus-minus models have existed for a long time in soccer. Back in 2009, Elijah Miller of Climbing The Ladder2 was writing about unweighted goal plus-minus tallies and using them to analyze MLS players. In recent years, plus-minus concepts have been formalized in various ways through academic papers, but the overall volume of these papers remains relatively low when compared to something like EPV. That is to say that despite the enduring allure of a plus-minus model for soccer, the metric remains a mystery to many, and popular public implementations of the stat have yet to emerge. In a way, RAPM is akin to a unreachable treasure; it's lack of true applicability for soccer represents the elusive nature of our sport. RAPM remains unsolved not because there has been no appetite to solve it, but because soccer is too unpredictable, too interdependent, too complex, too continuous.
But maybe this generalization is missing something. Maybe it has been solved. Maybe the issue was never RAPM, but instead how it was built. Maybe it just needed a little help.
In 2018, eight years after APM became RAPM and ridge regression doubled the model's accuracy3, that same advancement came to soccer. In addition to ridge regression, Matano et al. looked at RAPM through a bayesian lens and added priors based on player FIFA ratings, with the FIFA-informed RAPM model performing better than just the FIFA ratings by themselves and the standard RAPM model. FIFA ratings, however, do not exist for all players, and can often be finicky. Future research used box score statistics in addition to FIFA ratings for the prior calculation, which yielded better success. Other, simpler evolutions to RAPM involved the inclusion of xG difference as the response rather than goal difference, which creates more granularity in the model.
These days, most RAPM models utilize SPM for prior calculation, which takes box score stats and uses them to predict a base RAPM. SPM — you can think about it as xRAPM — is then used in a second RAPM run as a prior, essentially incorporating an on-ball ability proxy as a weight for assigning plus-minus responsibility. A recent paper went beyond true RAPM and incorporated a second metric — BPM (beyond plus minus) — in addition to SPM to create a composite metric. As the authors define it, BPM is based on the residual between stint-summed team xRAPM and real team xG differential. The resulting difference is used in a second ridge regression model that assigns BPM to individual players in that stint, without using performance context. The sum of BPM and SPM — EPM4 — is then used to compute league differentials. Since RAPM is theoretically isolated from team quality (obviously that's not really possible), a large enough sample should create stable league quality proxies.
Prior to league translations, certain leagues have naturally higher EPM values compared to other leagues. When this effect is removed, the values become far more stable. Lastly, the paper incorporated quantile scaling to reduce outliers, which further normalizes values in leagues that are considered "top-heavy" — think Ligue 1 and Bundesliga. The combination of all of these factors results in what the authors call GPM5, which goes beyond the true RAPM structure and instead builds a more robust metric using similar principles. The authors show that GPM is far more stable by position than VAEP, which is an incredibly useful finding:
This brings up a broader question of how these various metrics fit together and where RAPM, or SPM, BPM, EPM, GPM, etc. should go in the analytics toolbox. In reading papers and talking with fellow analytics people, the position I've come to is that RAPM can, and probably should be, used in conjunction with something like g+ or VAEP to get a better understanding of a player's impact. RAPM can in some ways go beyond on-ball metrics and the purely quantifiable, and instead can tell us more about what the player did off the ball, how they impacted the game with their presence. It isn't perfect, but it's context, and context is good.
Perhaps, then, RAPM shouldn't get such a bad rap (sorry). It requires a good enough data sample to be stable — in talking with people who know a lot more about this than me, it seems that 1.5-2 seasons across multiple leagues is enough — and a lot of work needs to be put in to ensure that the plus-minus model itself has the right context to attribute responsibility. It's not easy, and the constraints around such a model make it less flexible when compared to something like EPV. But this isn't a game of EPV vs RAPM. It's a game of information, and each model tells you a different thing. RAPM captures things that EPV doesn't, and the same is true in reverse. If I'm a decision-maker at a club, I want as much info as I can get, so building both and not just one of these models is important from the perspective of an analyst.
So, is RAPM solved? I don't think so. But I don't think that EPV is solved either. A far more interesting question is: what information can be gleaned from using RAPM and EPV together? Who gains value? Who loses it? Who is underrated by both models, and why? What information remains undiscovered, and how can we find a way to incorporate it?