Analysis: Going beyond rugby statistics, Part Two

Date published: May 30 2018

Our analyst Sam Larner is back this week, bringing readers the second part of his findings as he looks outside the box at rugby statistics.

Last week I introduced a slate of new stats looking at line-out throwing ability and a couple of different ways of measuring carry success. Hopefully you enjoyed that different way of assessing success in rugby. If you didn’t enjoy it I’m afraid there’s more of the same in this article.

I was helped this week by Beck Williams, she is a rugby player and programmer from Seattle who was able to turn her abilities to make the calculations required to create these metrics automated. I personally think I’ve saved the best stats until last and today we will be looking at arguably the three most important skills in rugby; kicking, passing and tackling. We will also be using the stats to make some practical decisions, specifically picking the English back-row. Finally, a word of warning, I will be showing how the data suggests that Don Armand might not be the answer to all England’s woes, I do know that this is heresy.

Passing Success Index (PSI) & Weighted Passing Success Index (WPSI)

It shouldn’t be a surprise that passing is really important in rugby. Every pass that is less than perfect will negatively impact your ability to beat the defence and keep hold of the ball for prolonged periods of time. Up until now however, we haven’t had the data publicly to know how good someone was at passing the ball. We have known how many times someone has passed and how many try assists they have but we haven’t had anything on the negative side of that equation.

The data set that I have been working with includes a number of other categories of passing ability, which has enabled these two stats to be created. PSI is really a percentage; successful passes divided by total passes. To give you a benchmark, of the 165 players on the list, all players with 50 or more passes, the low point is 73.1%, the average is 91.7% and the upper limit is 99.1% – all players are good at passing basically.

Now, you can use just PSI if you want, it corresponds pretty well with WPSI – the top five players on PSI are also the top five on WPSI for example. What WPSI tries to do however is treat each successful/unsuccessful pass differently. A try assist is better than a key pass and an intercepted pass is worse than a forward pass, we weigh each one according to how bad/good they are.

A quick heads up, these rankings come in the form of indexes, I explained this in more detail last week but just know that 100 is average, 110 is 10% better than average, 80 is 20% worse than average and 200 is twice as good as average.

Those top three names might be very surprising and I was absolutely shocked to see back-row player Sam Simmonds in the top 10, let along the top three. Hooker Mike Haywood is 14th on the list, but you have to go all the way down to Newcastle’s Ally Hogg in 48th place before you find another non-back. Simmonds has only passed the ball 52 times but he hasn’t had a negative outcome on a single one, not once – there’s not a single other person on the list who has achieved that. In fact, the only reason why Simmonds isn’t at the top of this list is because he has recorded one ineffective offload, the one misplaced pass all season, and because of his comparatively low total number of passes that drops him down to third. Of those 52 passes, three have been try assists and three have been key passes hence the third place in WPSI.

Joe Ford is probably a surprise to see at the top of the list as well, he benefits from a relatively small sample, just 112 passes but he’s had just one incomplete pass all season. It’s impossible to say, but I would be surprised to see Ford keep this up for an entire season.

Someone who doesn’t benefit from a small sample is Sam Stuart of Newcastle, he’s passed 486 times this season, his WPSI score really benefits from avoiding the serious negative results – intercepted pass and forward pass.

Faf de Klerk has passed more than anyone, 1,370 times, his PSI is 104.4 and WPSI is 123, both good enough for 39th in the list. His four intercepted passes leads the league although Luther Burrell has three in just 93 passes, that means his PSI is 103.4 (49th) but his WPSI is 68.9 (82nd).

It’s probably not a shock that scrum-halves dominate the top part of these rankings, 21 of the top 25, but the lack of fly-halves is surprising. After Ford, we have to go all the way down to the other Simmonds, Joe, who ranks 105.7 PSI (24th) and 185.6 WPSI (23rd). If you want a fly-half who isn’t called Joe then Dorian Jones and Marcus Smith are your next best shouts down in 30th and 31st by PSI respectively. If you’re wondering, George Ford is 102.9 PSI (54th) and Owen Farrell is 100.5 PSI (86th).

Finally, nine locks made the list, but only one – Elliot Stooke had a PSI above 100 (102, 62nd). Unfortunately, his WPSI was only good enough for an 84.53 WPSI.

Kicking Stats: Kick From Hand Index (KFHI) & True Goal kicking Index (TGI)

There are two quite different stats here; the first uses the same logic as the PSI and WPSI metrics – take the positive outcomes and divide them by all outcomes. It differs very slightly in that not all outcomes are either positive or negative for kicks but that doesn’t really have much of an impact. The second metric looks only at goal kicking, kick success is a well used stat but it doesn’t say much about how difficult a kick actually is. If someone makes 100% of kicks but they’re all in front of the posts are they a better goal kicker than someone who kicks 80% but with much harder kicks? Well, I can’t tell you that, you decide who you’d prefer to have, but I can tell you who has been the ‘best’ goal kicker.

Well done to anyone who has identified the fact that every single player in this list is a scrum-half. You might also have noticed that Saracens, Newcastle, Bath and London Irish all have two players in this list which is very interesting. The narrative is that Sarries kick a lot and kick well, and this seems to be played out here. What is probably more surprising is how good Sonatane Takulua is. The Falcons player has 15 positive outcomes from his kicks and just two negatives.

Freddie Burns in 12th is the best ranked fly-half on the list scoring 158.1. He’s the third of four Bath players in the top 20, most of any team, Rhys Priestland finishes the set in 15th with a score of 150.9.

De Klerk once again has the record for most kicks and his kicking is similarly average to his passing, 106.8, while surprisingly, amongst the lowest ranked 20 players there are the likes of Willie le Roux (49.1), Elliot Daly (29.4) and Henry Slade (23.6).

Stephen Myler also has the best kicking ability by success percentage, 88.5%, he does so well on this metric though because his average miss is from 42.8m which is the highest of anyone. Chris Pennell is the darling of this metric, his success rate of 70.5% wouldn’t get him anywhere near any leaderboard, however, his average made kick distance is 31.8m, quite a distance further than anyone else. Francis has a right to feel aggrieved; his success percentage of 82.1% beats six players above him on the list but a low average make and miss distance hurts his chances.

Owen Williams kicks long, his average miss is 39.11m, but his success rate of 53.3% can’t be overcome. To come top of this list with a success rate of 53.3%, Williams would need to average at least 50 metres in both his successful and unsuccessful kicks.

Gareth Steenson is successful 86.7% of the time, but his average unsuccessful distance is just 32.8m which is the second lowest on the list, behind only Joe Simmonds.

Defensive Ability Index (DAI)

Missed tackles and tackle success rates are some of the numbers that, even if you’re not a fan of stats, you’ll be familiar with. As we discussed last week, making tackles is a good way of measuring work rate but it’s not the best way of measuring defensive ability. In order to measure this we need something more than just tackles made or tackles missed, we need to know how good each tackle was, and we can do that with this data set which ranks tackles as dominant, complete or passive. All we then need to do is take all the good stuff, including turnovers, and divide it by all the stuff. However, in defence you can’t just make good tackles you also have to make lots of them, so we’ve factored in work rate as well.

If you want to be the best defender in the league it helps to be playing at Bath or Sale, six of the top 10 play for one of those clubs. There’s no sign of any backs in this list, as you would expect, but what may be a surprise is the inclusion of prop Marty Moore in 10th. The Irishman, who is heading to Ulster for next season, has made 106 tackles all season and just eight have been passive.

The work rate score is done per minute so you don’t need to have played a load to do well – Luke Wallace has made just nine appearances – but credit should be given to Jono Ross and Lewis Ludlow who have both played in every single game.

The two recently injured England players; Beno Obano and Jack Willis both rank highly in this metric, Obano is 27th (161.6) and Willis is 17th (174.2).

To find the best back we have to go all the way down to 70th and the London Irish centre Fergus Mulchrone. The Manchester-born player scores 137.3, thanks in some part to his three won turnovers.

No scrum-half is ranked above average on the index but Kahn Fotuali’i is the closes to that hundred mark, 98.6. The Samoan’s three sack tackles aren’t the highest for a nine, that title goes to Nic White of Exeter, however his 13 turnovers are the best by quite a margin.

Despite the press coverage, Armand doesn’t come out of this metric looking all that good, he’s the 150th best with a score of 107. His tackle success percentage is good but he suffers when we measure his work rate per minute. His score is good enough for 14th in the Exeter team. Armand also doesn’t do that well when his passing is taken into account, he’s mid pack of all back-row players there although he does come in towards the top of the different carry metrics.

Picking the England Back-Row Statistically

Okay, so ideally stats should be usable as well as informative. In this final section I’ll pick an English back-row using only the statistics for this season.

Blindside flanker – I personally like my blindside flanker to be able to contribute to the turnover game and carry a little bit. I created a stat to find the players who were good ball carriers and could turnover the ball. That brought up Simmonds but he just ranks 41st in defence. After a few non-English qualified players, Jono Ross is the next on the list and he was also the second best defensive back-row so he gets the nod.

Jono Ross – Sale Sharks – 6’3 – 110kg

Openside flanker – I want an openside flanker who can contribute in the attack and win turnovers and I’m willing to sacrifice some defensive ability for that. Simmonds again was the obvious choice but his four turnovers were a little concerning. Mark Wilson was another contender and was about to get in but I’ve plumped for the slightly better turnovers of Teimana Harrison, both he and Wilson have similar profiles for the other aspects of the game.

Teimana Harrison – Northampton Saints – 6’2 – 102kg

Number eight – Ball carrying and work rate are all I want from my number eight. Ben Morgan ranks highly for defence and work rate but he’s not what I need in terms of carries. Thomas Waldrom has the opposite issues. Nobody fully fills the job description but Zach Mercer does enough. His average gain is 3.2 metres – the second best for number eights behind Taulupe Faletau – and he’s also third for number eights according to DAI, good enough for 40th overall.

Zach Mercer – Bath – 6’3 – 105kg

Caveats to mention here are that some players, such as Billy Vunipola, probably would make this team but haven’t had the game time in the Premiership last season to count in these stats. For some of the tie breakers above I did look at other factors; such as line-out ability or penalties but these haven’t really been included as thoroughly as I might have liked.

Conclusion

Hopefully you’ve enjoyed these two articles looking at some extra data and metrics in rugby. As mentioned last week, the plan wasn’t to completely change how we view rugby, but more to pick out some important information, which isn’t currently captured in the data we have available. It shouldn’t be any surprise, for any of you who have watched rugby for the last decade or so, that there are more and more stats available without needing to dig too deep. This is the relentless rise of stats and hopefully we will begin to see more stats out there that reflect what is actually happening in the sport.

by Sam Larner