Hockey’s Analytics “Aha” moment?

Cam Charron has an interesting post on the state of Hockey Analytics over at The Score and how hockey executives are a step behind the hockey analytics bloggers but I have to disagree with one statement that Charron made.

There’s a reference in the Friedman piece to Craig MacTavish walking around looking for the “Aha!” moment when it comes to hockey analytics. I don’t think MacTavish has realized that half the hockey world is a step ahead of him in that regard. The “Aha!” moment comes when you realize that shots are a hell of a lot more predictive than goals for determining future events. As soon as you realize that hockey is a game between two teams trying to take shots on goal, I think the rest of it falls into place.

The problem with that thinking is that the minute we think hockey is all about shots and not goals the whole system could fall apart.  We know that shot quality exists. It’s a fact of life. A 45′ shot is generally not nearly as tough as a 10′ shot. A shot from 20′ after a cross ice pass is more difficult than a shot from 20′ on a two on two rush before the guys turn back for a line change. A screened shot from the point is more difficult than an unscreened shot from the point. Shot quality in that sense exists and is undisputed. The only reason shot analytics work is if over a large enough sample the quality of shots averages out such that the average quality of shot for one team is more or less equal to the average quality of shot for another team. I differ with some the extent that this is the case, but for this discussion I’ll go along with that premise. Now the problem is, when hockey starts to incentivize shots rather than goals I am not certain that that premise will hold up. There are lots of time a player could shoot the puck, but chooses not to because it is not a good scoring chance. If we start rewarding players on the basis of shot totals and that player starts shooting in those bad scoring chance situations the premise by which shot analytics is based on falls apart. Hockey at its core is, and always will be, about scoring goals. The fact that shot differentials correlate highly with winning is an interesting observation, and maybe even a useful one, but to change the focus of the game to shot differentials from goals differentials is not likely a strategy that will work in the long run.

Positive shot differentials is a result of good play and not because a team chose shot differentials as their goal and achieved it. The reality is, to generate positive shot differential you need to:

  1. When you have control of the puck you generate an offensive opportunity from that puck possession more frequently and you give up control of the puck less frequently.
  2. When you do not have control of the puck you force the opposing team to give up the puck more frequently and generate an offensive opportunity less frequently.
  3. You gain possession more frequently than the opposition be that through winning face offs or winning the puck battles after shot attempts.

If you can win the puck battles, give away the puck less frequently and force the opposition to turnover the puck more you should win the shot differential contest. I suspect shot differential is highly correlated with winning because good teams do those three things better than their opposition and not choose to shoot more often than their opposition. We really need metrics to measure those three things but unfortunately we don’t have them. The work being done on carry the puck into the offensive zone vs dumping the puck in is valuable because it hits at the heart of those good attributes (i.e. what is the best way to generate an offensive opportunity when we have possession of the puck).

This isn’t to suggest that looking at shot totals is a bad thing. So long as we live in a world where driving shots is not the primary goal, shots totals can act as a proxy for identifying players who might have some of those other good attributes and since we have no good metric for measuring them. We just have to be careful that we aren’t identifying systems that result in more shots but not more good shots. Again, shots is not the goal, goals are.

Furthermore, it is quite possible that shot differential analytics can result in a value proposition for GMs. In my post last week about the declining predictive value of corsi/fenwick I showed that as sample sizes increase corsi/fenwick does a poorer job of predicting future events at the team level than with smaller sample sizes where as the percentages and goal metrics maintain or improve their predictive value. In that post I deliberately was careful about drawing any conclusions about what it meant because, to be honest, I am not completely sure what it means though I do have a couple of theories. One is that it could mean that corsi/fenwick is largely driven by the depth of the team and for many teams the second and third lines have a fair bit of turnover over the course of 2-3 years (where as elevated shooting percentage or save percentage is largely driven by the elite players who don’t change teams nearly as often). If GMs aren’t evaluating second-tier players using shot differential metrics they may not be replacing the players with similarly talented (shot differential-wise) players. If this were true, it could mean that this is a flaw in current thinking and that a smart GM could exploit this flaw but again by filling his second and third lines with positive shot differential players. This could give his team the depth it needs to win. It is just a theory but one worth exploring more.

In the end though, hockey is all about out scoring the opponent, not out shooting them. Always has been, always will be, and that is the way it should be. Realizing that that shot differentials is highly correlated with winning is not the ‘aha’ moment in the sense that all of hockey should change focus to out shooting over out scoring at the cost of shot quality because that won’t work. The focus always has to be how to generate more shots from good scoring plays, not just generating more shots.


This article has 7 Comments

  1. I definitely don’t disagree that certain shots have different values, but players with high Corsi ratings aren’t just shooting the puck from outside areas to boost shot counts.

    Long range attempts never get inside for dangerous opportunities. Good Corsi players use a variety of weapons to get multiple shots and chances per possession in the zone. I think when you’re continuously firing pucks from dangerous areas, it’s harder for the goalie to control, harder for the defense to block, and more likely the offence will get more shots.

    It’s the repetition over games where you start to notice the difference, not on the individual plays. I tallied up Leafs scoring chances yesterday and the results sync up a lot more with Corsi than with +/-.

    1. I agree that players with high Corsi ratings aren’t just shooting the puck from outside areas to boost shot counts, but that is because shot counts largely aren’t incentivized, goal counts are. As soon as you switch the incentive from goals to shots (which in essence is what you are claiming should happen), can you be assured that that won’t be the case? That’s the point I was making.

      We disagree on the value of corsi (I do believe players, and to some extent teams, can drive shooting percentage) so I am not sure that scoring chances syncing up with corsi is a good thing for the credibility/value of scoring chances but I have avoided getting into this debate because I know how much work people put into tracking scoring chances. I just think that humans just aren’t good enough at identifying the difference between a great scoring chance and a good one and a poor one. The difference could be a fraction of a second delay in getting the shot off or a fraction of an inch with respect to the positioning of the pass.

      1. If we start rewarding players on the basis of shot totals and that player starts shooting in those bad scoring chance situations the premise by which shot analytics is based on falls apart.

        This is Goodhart’s Law — in economics it roughly states that once a measure becomes a policy target, it ceases to be a useful measure, for reasons of incentives, just as you say. It’s a well-known result in the social sciences and it’d be good to see more awareness of it in sports statistics.

    2. Do I recall reading that you had recorded scoring chances for the Leafs’ games in February, wherein you discovered that despite the Leafs taking fewer shots, they out-chanced (and defeated) their opponents?

      It may only have been a span of 4 or 5 games, but those examples surely counter yesterday’s example.

    3. “Good Corsi players use a variety of weapons to get multiple shots and chances per possession in the zone.” you say. Tavares and Kane currently lead the league with 99 shots in somewhere over 550 shifts. Better get your statistical calculator out and figure out just how many multiple shot possessions either one has probably had. (Sorry for the snarkiness, but that was a completely ridiculous statement.

  2. I I must be missing something, but what I take from the Charron quote is that teams should be looking for ways to improve their shot totals, especially relative to their opponents. To do that, you need to examine all of the things that you talk about: what is the best way to get shots, get control of the puck in ways that give us opportunities for those shots, and minimize our opponents ways to do the same. I see your thoughts simply as fleshing out Cam’s.

  3. David;

    Your theory is sound..but all my efforts to find the ‘best’ predictor for playoff results…finds that Fenwick #’s out perform all
    other stats…GD/ Even GD/ Sve% GF GA etc? this is over ~400 games? Any ideas?

Comments are closed.