Let me start off by first saying that this isn’t going to be a research post as much as it will be a commentary on the past, present and future of shot quality research.
I have had more than a few battles on shot quality so I feel I have a more than decent understanding on the subject. As outlined in this post by Michael Schuckers there are two aspects of shot quality. These are, the quality of an individual shot and the average shot quality of all shots taken by a team or a player when on the ice.
Individual Shot Probability does matter and this has been illustrated time and time again. There’s no doubt about it. The most recent example is the distance analysis by Michael Parkatti, Different shots have different probabilities of going in and there are plenty of factors that influence these probabilities. These include x and y coordinates as well as the type of shot matter. Here are some heat maps to emphasize that.
What has not been shown to matter much, to my mind, is Average Shot Probability (ASP), either for shots that a goalie has faced or that a team has faced or that a team has generated over a long period of time. It might be there but the consensus (yes, David, I see your hand is up) is that it is not. I’ve tried to look for it. It, ASP, matters but not a ton. I’ve got plans to take another look at it again this winter. But there is little denying that where we are right now is that we lack evidence for the value of long term repeatable ASP. Somewhere there’s a fourteen year old kid with mad R skills and a great idea on how to model these data and, perhaps, they’ll find that shot quality exists. It’s just that right now we don’t have enough evidence for it.
Yes, the “David” with his hand up is me. I have always claimed that shot quality exists in the Average Shot Probability sense of the word. I believed it back then and I believe it now. The reason I believe it is the data supports it as there is clearly a difference between the on-ice shooting percentages of the players at the top of this list and the players at the bottom and the players at the top are mostly who we all consider as the elite offensive players in the league and the ones at the bottom are mostly 3rd and 4th line players. This isn’t coincidence or randomness and is the strongest evidence in support of shot quality we have. There are even teams that have consistently posted above or below average shooting percentages. Shot quality in the Average Shot Probability sense exists and we must acknowledge that.
A number of people have analyzed shot location or shot distance data (including Shuckers, myself and numerous others) and have found relatively little indication that shot location varies across teams in a significant enough way to have a significant impact on shooting percentage. This does not mean that Average Shot Probability does not exist (which Shuckers implied was the consensus) but rather that that Average Shot Probability is not significantly influenced by variations in average shot locations. There is an important difference and the latter does not mean that Average Shot Probability does not exist (I think this is the crux of many of my shot quality debates in the past like those with Gabe Desjardins).
One of my favourite articles on this subject is one written by Tom Awad in his “What makes good players good” series of posts. If you haven’t read this article I recommend you go read it now. It is probably the best article written on shot quality even though it isn’t explicitly about that. The most important thing to note is the last table which I will reproduce here:
Group +/- due to finishing +/- due to shot quality +/- due to outshooting 1st tier 0.22 0.04 0.15 2nd tier 0.07 0.02 0.10 3rd tier 0.00 0.01 -0.06 4th tier -0.20 -0.04 -0.15
In this table ‘finishing’ is essentially having a better shooting percentage than your opponents, ‘shot quality’ is having a better average shot location and outshooting is as it sounds, out shooting your opponents. The greatest spread in talent between first tier players and fourth tier players is being able to out finish your opponents followed closely by outshooting your opponents. Having a better average shot location is a relatively minor factor in what makes good players good. A key takeaway is average shot location has relatively small impact on average shot probability which is consistent with what everyone has found. This is the “consensus” that Shuckers is talking about.
War-on-ice has recently come up with a new definition for a scoring chance and added the results to their statistical database. The definition starts with the notion of “danger zones” which are areas surrounding the zone in front of the goal (similar to the “home plate” definition) with additional adjustments for rebound shots and rush shots (which are based of my work from this past summer). Their formal definition of a scoring chance is as follows:
- In the low danger zone, unblocked rebounds and rush shots only.
- In the medium danger zone, all unblocked shots.
- In the high danger zone, all shot attempts (since blocked shots taken here may be more representative of more “wide-open nets”, though we don’t know this for sure.)
This definition, while likely an improvement over anything we have had previously (and there is evidence to support that), it still significantly dependent on shot location and based on the history of not being able to find much of a link between shot location and shot quality I have concerns about it. In particular, are we watering down the definition of a scoring chance by relying too much on location? Might we get better results by looking at just rush and rebound shots? Until I see a formal analysis that shows that shot location is a major factor in Average Shot Probability at either the team or player level I have my doubts that using shot location when defining what a quality scoring chance is is beneficial (and may in fact by harmful by diluting the defnition).
Some other really interesting work being done recently is by former NHL goalie Stephen Valiquette where he is identifying higher quality shots as being those that (for the most part) result from plays with significant lateral movement. In particular he defines the “Royal Road” as the line down the middle of the ice from one end to the other and when the puck moves laterally across this line either by a pass or being skated across immediately before a shot is taken the shot is more likely to result in a goal. To me this makes a lot of sense and I think is really where the next great leap in shot quality analysis will come from. Speed of the play (i.e. rush shots) and lateral puck movement are likely the largest contributing factors to shot quality.
In support of the idea that puck movement is a significant factor in shot quality a couple of years ago I looked at the relative impact a player can have on his linemates shooting percentages and found that many of the best players at boosting line mate shooting percentage are excellent playmakers.
The challenge with Valiquette’s “Royal Road” work is that it currently requires a lot of manual tracking of the data which is time consuming and has the potential to bring human error into the analysis. Furthermore it also doesn’t account for speed of the play which may also be a significant factor in shot quality as it limits the goalies reaction time. While I believe Valiquette’s work is a significant step forward in our understanding of the game the holy grail of shot quality research will be when the NHL introduces player and puck tracking technology. When we get this data we will be able to dig far deeper into shot quality research and allow us to define shot quality in far greater detail. Everything that has been done up until now will pale in comparison to what we’ll be able to do with automated player and puck tracking data.