I was thinking the other day about the metrics we use to evaluate team performance in football/soccer and I thought I’d write a little about it. Now, definitionally SoTR, TSR and other such metrics (definitions here) are comparative; it’s easy when discussing them as a proxy for team quality to forget this, but it’s important to remember when we quote a TSR/SoTR we are looking at a team’s ability to generate shots relative to their opponents. TSR or SoTR are by no means absolute measures, because the teams that make up each division change each year.
I think this is in part why the various shot ratios are much less repeatable from year to year in the Championship than in the Premier League; the Premier League is a more rigid league than the Championship, changing only 3 teams each year combined with a financial structure that helps to maintain the status quo (i.e. there is more disparity in terms of team quality by default).
This brings me on to my second thought. Given we expect a degree of noise around the mean, the relationship between the various shots ratios and points ought to be clearer when the league is more skewed ( so long as the noise increases at a lower rate than the range in SoTR does, which I think is reasonable). The following gif shows the plots of SoTR vs Points from each season from 2004-2005 along with the least-squares trend-line from the combined data (i.e. the data from all years). The number on the bottom is the r-squared (a measure of how well the model matches the data: explanation here) for that year’s data and the line fitted from the combined data.
So there’s perhaps a couple of things we can look at from this.
- 2012-2013 was weird. With such a tight range of SoTRs, the relationship between SoTR and Points becomes difficult to see. The negative r-squared tells us that for this specific data set, a flat line (i.e. SoTR doesn’t have any effect on Points) fits the data better than our overall trend-line. 2013’s weirdness may be worth a closer look at some point.
- It does appear that my initial hypothesis that the relationship is clearer when the league is more stretched (I suppose r-squared could be interpreted as a measure of signal:noise). Moreover, I have another graph that appears to back this up:
While I don’t want to fall into the trap of simply throwing out statistical measures without thought, I think this plot looks pretty encouraging; there appears to be a clear linear relationship between how well SoTR correlated with Points scored over a season and how uneven the league is that year.
I started this post recalling that shots ratio-based metrics are comparative metrics. As a result, a team which stays at the same quality from one year to the next will not necessarily have a similar SoTR the next year. Likewise I should be wary of comparing two teams across different years directly by SoTR or TSR without context (although in truth any statistical analysis without contextual awareness is pernicious). Likewise, it does appear that SoTR/TSR are more powerful metrics for evaluating relative team quality in years where the league is more uneven.