Evaluating SCF and HSCF and our ability to quantify shot quality

This past season War on Ice introduced two new shot quality metrics – Scoring Chances (SC) and High Danger Scoring Chances (HSC) which are defined here.  Stephen Burtch has previously evaluated this scoring chances with respect to their ability to predict future goal scoring and goal differentials and found them to be a better predictor than traditional possession statistics. As a strong believer in shot quality I am not surprised by this conclusion but with this post I want to take a closer look at really how well these metrics are at measuring shot quality.

The premise underlying this analysis is a simple one. The higher the percentage of overall shots (or shot attempts) that are scoring chances or high danger scoring chances the higher the likelihood the player will post a higher shooting percentage. So, I will evaluate the following “on-ice” relationships:

  • HSCF/CF vs CSh%
  • HSCF/SF vs Sh%
  • SCF/CF vs CSh%
  • SCF/SF vs Sh%

To do this evaluation I will take all single seasons for all forwards with at least 500 5v5 minutes in that season for the past 8 seasons. All totalled there were 2611 such player seasons. Here are the results:

HSCF_vs_CShPct

HSCF_vs_ShPct

SCF_vs_CShPct

SCF_vs_ShPct

There are two key takeaways from these charts. First, these metrics do positively correlate with shooting percentage so to some degree these metrics are capturing shot quality though the correlations aren’t particularly great. The second takeaway is that considering all scoring chances is better at measuring average shot quality than restricting to just high danger scoring chances.

There is another metric that is known to correlate well with shooting percentage for forwards though. Ice time. To see how these scoring chance metrics stack up against ice time I looked at the relationship between TOI% (or the percentage of the teams ice time the player gets) and shooting percentage.

TOIPct_vs_ShPct

Now that is interesting. TOI% is quite a bit better at estimating shooting percentages than our shot quality metrics. This tells me two things. First, while scoring chance data is telling us something about shot quality there is still a lot that isn’t accounted for. Second, scoring chance data hasn’t caught up to the coaches “eye-test” ability yet – coaches are better at evaluating talent than our scoring chance data.

The last thing I want to look at is a larger sample size so I took all forwards who have played at least 3000 minutes over the past 8 seasons combined – a total of 364 players. Here is the SCF/SF vs Sh% chart.

SCF_vs_ShPct_8yr

Going with the larger sample size improves things a fair bit. For interest, the r^2 for HSCF/SF vs Sh% using the 8-years of data is 0.21 so scoring chance data is still much better to use than high danger scoring chance data.

So, how did the coaches do over the 8-years at handing out ice time? Pretty good.

TOIPct_vs_ShPct_8yr

In conclusion, shot quality is still something we are having a terribly difficult time understanding. It clearly exists and is a significant factor in driving on-ice results but out ability to measure and quantify what leads to higher shot quality is still clearly lacking. Scoring chances as defined by War on Ice might be a step in the right direction but our attempts to quantify it are still a step (or two) behind the coaches.

 

This article has 1 Comment

  1. Great work David. I’ve seen Leafs fan argue that Jake Gardiner is a stud because he’s got impressive HSC and SC rates. And that his poor GF/60 rate and CSh% within the past 3 seasons (a 171 GP sample size) is really a result of bad puck luck. But when you’re looking at 171 games of data and Jake is clocking #2D minutes on the 5v5, is it *really* bad puck luck? Or do the HSC and SC stats fail to properly identify actual high-danger scoring chances and scoring chances? Jake is good at taking the puck up the ice. But if his passing and creative vision is so great, why aren’t his passes generating more goals across a sample size of 2907 minutes 5v5? Jake Gardiner ranks within the top 90 D (2801+ mins) in 5v5 TOI in the last 3 seasons. Among that cohort, he ranks 64th/90 in pts/60, 22nd/90 in goals/60, 80th/90 in assists/60, 66th in first assists/60. But he ranks 40th/90 in IPP, 12th/90 in IGP and 66th/90 in IAP. I’d wager that his IFAP and Individual Primary Points Production rates would rank higher than IAP and IPP because he’s better at the first assist game than second assists.

    You could argue that Jake Gardiner plays with forwards with weak shooting percentage and that’s why his assist rate and GF/60 rate isn’t so good. But his TMGF/60 rate is a lot higher than his GF/60. Jake Gardiner is a good puck-moving defenseman but he’s not the playmaking prodigy that analytics fans think he is. And for an “offensive defenseman”, he sure stinks on the power play. Jake Gardiner doesn’t have the hockey IQ to work a power play. He doesn’t have the hockey IQ to actually make good plays consistently. Even though the scoring chance metrics say he does. He scored at a 0.4 PPG clip in his rookie season and received a lot of hype for this. And despite increases in his PP time, he has yet to match, let alone surpass his rookie PPG rate after playing 3 full pro seasons in the NHL and one short season. I think Jake Gardiner’s concussion with the Marlies in late 2012 might have affected him. If I were to include Jake Gardiner’s rookie season and look at 4 year data, his numbers improve. But if you can’t get back to where you were 3 years ago, a rookie season, something smells.

Comments are closed.