Evaluating SCF and HSCF and our ability to quantify shot quality
This past season War on Ice introduced two new shot quality metrics – Scoring Chances (SC) and High Danger Scoring Chances (HSC) which are defined here. Stephen Burtch has previously evaluated this scoring chances with respect to their ability to predict future goal scoring and goal differentials and found them to be a better predictor than traditional possession statistics. As a strong believer in shot quality I am not surprised by this conclusion but with this post I want to take a closer look at really how well these metrics are at measuring shot quality.
The premise underlying this analysis is a simple one. The higher the percentage of overall shots (or shot attempts) that are scoring chances or high danger scoring chances the higher the likelihood the player will post a higher shooting percentage. So, I will evaluate the following “on-ice” relationships:
- HSCF/CF vs CSh%
- HSCF/SF vs Sh%
- SCF/CF vs CSh%
- SCF/SF vs Sh%
To do this evaluation I will take all single seasons for all forwards with at least 500 5v5 minutes in that season for the past 8 seasons. All totalled there were 2611 such player seasons. Here are the results:
There are two key takeaways from these charts. First, these metrics do positively correlate with shooting percentage so to some degree these metrics are capturing shot quality though the correlations aren’t particularly great. The second takeaway is that considering all scoring chances is better at measuring average shot quality than restricting to just high danger scoring chances.
There is another metric that is known to correlate well with shooting percentage for forwards though. Ice time. To see how these scoring chance metrics stack up against ice time I looked at the relationship between TOI% (or the percentage of the teams ice time the player gets) and shooting percentage.
Now that is interesting. TOI% is quite a bit better at estimating shooting percentages than our shot quality metrics. This tells me two things. First, while scoring chance data is telling us something about shot quality there is still a lot that isn’t accounted for. Second, scoring chance data hasn’t caught up to the coaches “eye-test” ability yet – coaches are better at evaluating talent than our scoring chance data.
The last thing I want to look at is a larger sample size so I took all forwards who have played at least 3000 minutes over the past 8 seasons combined – a total of 364 players. Here is the SCF/SF vs Sh% chart.
Going with the larger sample size improves things a fair bit. For interest, the r^2 for HSCF/SF vs Sh% using the 8-years of data is 0.21 so scoring chance data is still much better to use than high danger scoring chance data.
So, how did the coaches do over the 8-years at handing out ice time? Pretty good.
In conclusion, shot quality is still something we are having a terribly difficult time understanding. It clearly exists and is a significant factor in driving on-ice results but out ability to measure and quantify what leads to higher shot quality is still clearly lacking. Scoring chances as defined by War on Ice might be a step in the right direction but our attempts to quantify it are still a step (or two) behind the coaches.