Question for those who don’t believe in shot quality

I am a firm believer in shot quality.  I have probably looked at it a dozen different ways and it seems pretty clear to me that it exists and yet there are still a lot of doubters out there so I wanted to take yet another opportunity to show that show quality exists and give an opportunity for the shot quality deniers to tell me what I am doing wrong if I am doing something wrong.

First lets start off with a definition of shot quality to make it clear what I am referring to.  Some people refer to shot quality as shot location (i.e. shot distance) but to me shot quality is even more basic than that.  For me for shot quality to exist it must only be shown that one group of shots is more difficult to save than another group of shots after taking into account game situation (i.e. 5v5 vs PK vs PP), score effects, and any factors that might affect shot quality.

So, let me take 2 groups of shots.

Group 1:  All the shots taken by the Penguins when Sidney Crosby is on the ice during 5v5 close zone start adjusted  (eliminating first 10 seconds after a face off) play over the past 5 seasons.

Group 2:  All the shots taken by the Penguins when Tyler Kennedy is on the ice during 5v5 close zone start adjusted  (eliminating first 10 seconds after a face off) play over the past 5 seasons.

I am using 5v5 close situations so that eliminates any situation and score effects that might influence shot quality.  I am also eliminating the shots within 10 seconds of a face off because they have been shown to be far easier shots to save and we wouldn’t want to disadvantage one above groups if it had a disproportionate number of shots immediately after a face off.

Also, since both players have played the past 5 seasons with the same team it eliminates any scorekeeper bias with their shot counts since both have played behind the same score keepers.  So, let’s look at the numbers.

Group GF SF Sh% Chance
Group 1 (Crosby) 150 1292 11.61% 1.65%
Group 2 (Kennedy) 92 1166 7.89% 1.23%
Totals 242 2458 9.85%

The Sh% column looks pretty telling on its own, but the Chance column says it all.  The Chance column is the liklihood that the groups shooting percentage could be what it was based on luck alone using a binomial distribution with a real shooting percentage of 9.85%.  In short, it is very unlikely that Crosby and Kennedy would have those on-ice shooting percentages based on luck alone.

Truth be told, for these two players score effects would have very little effect on their relative stats since they played almost equal time leading as trailing so we could increase our sample size fairly significantly if we looked at 5 year zone start adjusted 5v5 shots and goals.

Group GF SF Sh% Chance
Group 1 (Crosby) 243 1964 12.37% 0.17%
Group 2 (Kennedy) 150 1837 8.17% 0.09%
Totals 393 3801 10.34%

So essentially there is no chance that that happens based on luck alone.  Shot quality of some sort is a factor.  There is definitely show shot quality skill/talent factoring into the equation.

So my question to shot quality deniers is as follows.  If the above is not evidence that shot quality exists, why not, and if it is evidence of shot quality, why are you still in denial that shot quality exists?


This article has 2 Comments

  1. Thanks for this.

    Seems like the term ‘Shot Quality’ is a big part of the issue. Seems to me (and I’m at the center of this thanks to DIGR) that the question is not about probability of individual shots or the average probability of shots by a player but rather about whether the average shot probability for a team can be controlled. Consequently, I prefer to think about this issue as not shot quality but as average shot probability. Certainly different shots have different probabilities of resulting in a goal but the larger question is whether or not teams can impact the average shot probability.

    As a footnote, I think that we have quite crummy models for assessing average shot probabilities (including the nonparametric one that I use for DIGR) and until we know things like whether or not a goalie is screened or the speed of shots then we are not going to see drastic effects. That does not mean we won’t see *significant* effects.

    1. Yeah, the term “shot quality” is a bit confusing and I think you have partly hit on the issue. Shot quality could apply to any number of things from the quality of an individual shot, to the quality of a group of shots such as the ones taken by an individual, the ones taken while a player is on the ice, the ones taken by a team, etc. We know shot quality exists in various forms. PP shots are on average of higher quality than 5v5 shots. Shot quality seems to vary by game score. Certain individuals can improve their individual shot quality based on their playing style (i.e. Mike Knuble taking a ton of close in shots because he stands in front of the goal most of the time). The one thing people have chosen not to believe in is that shot quality can and does exist at the team level or on-ice level and thus concluded corsi is the best metric for evaluating team and player abilities. Certainly at the player level this is not true (given sufficient data of course).

      As for as shot quality models, I think we are a long way from being able to get useful models. As you say, too many currently unmeasured factors go into determining shot quality from the shooting skill of the player, the speed of the shot, the positioning of all the players on the ice, how soon the shot was taken after a pass, how long that pass was (i.e. cross ice pass), how quick the pass was, etc. Maybe if we had automated player and puck tracking to track where the players and puck is at all times we could come up with some complex shot quality model but for now I think attempting to model shot quality is mostly a pointless exercise.

Comments are closed.