Advanced match performance metrics: a view from Australian rugby

From the Sports Analytics conference site is this article from Sydney's Daily Telegraph about the development of a new performance metric in Australian rugby league and its influence in the outcome of the recently completed National Rugby League season.

The metric is called the Contributor Value Rating (CVR) and is the product of NRL Stats, the official statistics company for the NRL.  It is a regression model of a collection of 50 in-match statistics, which is then weighted by the number of minutes played to calculate the final score. Some trade studies have been performed with the rating to establish the connection between score levels and team performance, and those studies show that there is a strong correlation between squad players with high CVRs and finals (playoff) appearances.

The rating found a champion in Manly Sea Eagles coach Des Hasler, who used the rating to determine which areas of his players' game needed improvement during the season, and New South Wales coach Ricky Stuart, who used the metric to select his state representative side.  The result was that Manly transformed from a team that was last-in and first-out of the playoffs to one that became NRL premiers (champions).

There are some lessons learned from this match performance metric that have applications for work done in other sports, including soccer:

  • Incorporate sport-specific knowledge.  Statistical analysis can be quite involved and does require a specialist, but the inputs into a model have to be connected back to relevant events in a match. In order to insure this it is best to use the expertise of those in the field, such as coaches and performance analysts.  I also liked that all of the coded video data was used — it's too much work to ask for a performance analyst if the data aren't going to be used in any way.
  • Metrics must have a predictive quality.  Admittedly, a hard problem, and an easy way for a quantitative analyst to lose his credibility.  Nevertheless there has to be some kind of calibration to final results in the league table for these metrics to be taken seriously.  What is the difference between a relegation candidate and mid-table?  Mid-table and top eight? Top eight and top three or four?
  • Enable identification of key performance indicators. In effect, close the loop between match performance and training.  This also closes the loop between measurement and training.
  • Find a champion. Very few NRL teams gave the CVR a look when it was developed — except for Hasler. He was in a moment of pain after an early Finals elimination, but he also trusted his performance analyst and bought into the process.

The CVR has been shown to be an effective part of a rugby manager's toolkit, but now that Manly have become champions their advantage might not last for much longer.