Mar 182011

The guys over at Behind the Net have initiated a ‘prove shot quality exists’ competition and in response to that Rob Vollman took a quick and dirty look at shooting percentage suppression.  As I showed the other day, Rob’s logic was a little off.

Rob started off by identifying a number of players with high on ice save percentages over the past 3 seasons.  Some of these guys included low minute players mostly playing on the fourth line against other fourth line caliber players, but there were a handful of players who played relative significant number of minutes and still put up good on ice save percentages.  Let me remind you of a few names that Rob identified:  forwards Marco Sturm, Manny Malhotra, Tyler Kennedy, Travis Moen, Taylor Pyatt, Michael Ryder, defensemen Kent Huskins, Sean O’Donnell, Mike Weaver, Mark Stuart.  I’ll get back to these guys later but I’ll claim that Rob dismissed some of them prematurely by claiming they played against weak competition.

As you may or may not know I have developed offensive and defensive ratings for every player and these can be found at Furthermore, I have created these using goals for/against as well as shots for/against, fenwick for/against, and corsi for/against.  For clarification, fenwick is shots + missed shots while Corsi is shots + missed shots + blocked shots.  For this study I decided to use fenwick instead of shots because I had the data handy and I was too lazy to get the shot data in the right format but there shouldn’t be a significant difference (the two are very highly correlated).

Continue reading »

Mar 162011

I have posted a few articles here recently about the existence of shot quality, one of which related to last seasons Washington Capitals and one related to how shot quality varies according to game score but there are still shot quality deniers out there.  One of the comments I received from a shot quality denier to those posts was as in depth as “You did it wrong” but offered no further explanation.  So there it stands.

Derek Zona and Gabe Desjardins over at Behind the Net Hockey (mostly shot quality deniers) have put up a $150 prize for anyone who can show that shot quality exists.  One method they suggested one could pursue to prove such a thing was the following:

Are there players or teams with the ability to drive or suppress on-ice shooting percentage?  What are their characteristics?

This prompted Rob Vollman (who I presume is a shot quality denier, my apologies if not) to look into just that and to do so he identified a group of players who had the highest save percentage against while they were on the ice.  The theory is, if shot quality suppression was a talent then there should exist players who experience a very good save percentage for their team while they are on the ice.  The group of players identified varied significantly from George Parros to Kyle Wellwood to Sean O’Donnell to Marco Sturm.  In the end Rob came to the conclusion that these players all had high save percentages while they were on the ice because they mostly played against weaker quality of competition.

But none of them are facing their team’s toughest minutes.  If they truly had the ability to suppress shooting percentage, why would Kesler and Burrows hop out against Ovechkin instead of Malhotra?  Why would Pronger keep an eye on Crosby instead of O’Donnell?  Kudos to each of them for playing their roles very well, but the explanation still appears grounded in Quality of Competition.

And there is the fault in logic.

  • Claim:  Shot quality doesn’t exist.
  • Counter-evidence:  Some players do experience higher save percentages while they are on the ice.
  • Rational:  They do so because they play against weaker quality of competition.
  • Claim Confirmed:  Phew, my claim that shot quality doesn’t exist remains valid.

Now the whole problem with that theory is the rational part because the rational part requires shot quality to be real for it to be true.  The only way you can have a better quality of competition (in terms shooting/save percentage) is to have shot quality exist.  If shot quality didn’t exist all competitors would have the same level of shooting percentage talent.  The claim and rational can’t both be true, so the logic fails.

And that is where identifying shot quality becomes difficult.  Players that are generally good at reducing the quality of shots against are lined up against opponents who are generally good at creating quality shots for.  The net result is their talents cancel each other out to some extent making it difficult to identify shot quality driving/suppressing talent just by looking at the numbers in isolation of who they are playing with and against.

Mar 152011

I thought this debate had been fully hashed out already but apparently some people still don’t believe that the game score has an impact on shooting percentage (and shot quality).  The following table shows the shooting percentages by game score over the past 3 seasons (2007-08 to 2009-10) during even strength situations where neither goalie is pulled for any reason (including delayed penalty situations).

Situation Shots Goals SH% Prob<= Prob>
Down2+ 23650 1852 7.83 0.3794 0.6206
Down1 30447 2356 7.74 0.1696 0.8304
Tied 60753 4427 7.29 0.0000 1.0000
Up1 26842 2288 8.52 0.9999 0.0001
Up2+ 19351 1779 9.19 1.0000 0.0000
Overall 161043 12702 7.89 0.5024 0.4976

The Situation, Shots, Goals, and SH% columns are self explanatory.  As you can see, shooting percentage is at its lowest in game tied situations, increases slightly for teams that are trailing and increases significantly for teams that are leading.

The second last column titled Prob<= show the probability (according to a binomial distribution) that that number of goals or fewer would be scored on that number of shots if the expected shooting percentage was 7.89%, the same as the overall 5v5 shooting percentage.  The last column titled Prob> is simply 1-Prob<= and shows the probability of getting more than that number of goals on that number of shots.  So, in down 2+ goal situations, there is a 37.94% chance of their being 1852 or fewer goals scored on 23650 shots which indicates that the down2+ shooting percentage isn’t different from the 5v5 mean at any reasonable confidence level.  The same conclusion can be drawn about down1 situations.  But, the shooting percentages in game tied, up1 and up2+ situations are statistically different at an extremely high confidence level.  Essentially there is zero chance that game tied, up1, or up2+ situations have the same natural shooting percentages as game overall 5v5 situations.  In no way can luck be the sole reason for these differences.

So, does this conclusively tell us that shot quality exists and varies according to game score?  It probably does, but I can’t say it is conclusive as it could mean that teams that trail a lot have bad goaltending (the reason they are trailing) and this results in the team leading having an inflated shooting percentage.  So, what if we looked at shots against a particular team.  Let’s say, for example, against the NY Rangers.  Here is what that looks like.

Situation Shots Goals SH% Prob<= Prob>
Overall 5159 386 7.48 0.5135 0.4865
Up1 843 73 8.66 0.9116 0.0884
Up2+ 485 46 9.48 0.9571 0.0429
Leading 1328 119 8.96 0.9800 0.0200
Tied 2004 138 6.89 0.1658 0.8342

I chose the Rangers because they use predominantly one goalie and that goalie is generally speaking a quality goalie.  As you can see, the confidence levels aren’t quite as strong as league wide mostly because of the smaller sample size but if we combine the up1 and up2+ categories we can say that shot quality against the Rangers when the opposing team is leading is statistically different than shooting percentage against the Rangers overall.

If you are interested in seeing what happens with a team that has had chronically bad goaltending, here is the same table for the Maple Leafs.  We see the same sort of things.

Situation Shots Goals SH% Prob<= Prob>
Overall 5309 491 9.25 0.5120 0.4880
Up1 938 94 10.02 0.8098 0.1902
Up2+ 906 100 11.04 0.9698 0.0302
Leading 1844 194 10.52 0.9712 0.0288
Tied 1985 149 7.51 0.0034 0.9966

So what have we learned.

  1. Shooting percentages vary according to game score.
  2. Those shooting percentage differences can’t be attributed to luck.
  3. Those shooting percentage differences can’t be attributed to goaltending.

That means, it must be the quality of the shots that varies across game scores.  In short, we can conclude that when teams get down in a game they open up and take more chances offensively which in turn gives up higher quality shots against which makes perfect sense to me.

When we combine this with my previous post on the Washington Capitals shooting percentage last season, it is probably safe to assume that shot quality exists and we can’t safely assume that all shots can be treated equal in all situations.