Roles and Stats Part I: Background and Methodology

This is the first of what will be a series of posts on the impact of roles on stats with a particular interest in the impact of roles on shooting and save percentages.

Background

My involvement in hockey analytics has been somewhat outside of the main stream. I am a firm believer in shot quality and the ability of players to influence on-ice shooting and save percentages while hockey analytics has generally trended (almost exclusively) towards puck possession metrics like Corsi. As a result, over the years I have fought many battles with many people on shot quality starting first with shooting percentage. These old shooting percentage debates went somewhat like this:

Others: I ran [year over year/split half correlation/etc.] correlation and found little or no persistence in on-ice shooting percentage therefore on-ice shooting percentage does not exist (or at least is dominated by randomness and thus is not important).

Me: But why do first line players all seem to have shooting percentages significantly higher than all the third line players?

Others: Those shooting percentages are unreliable due to the randomness in them. You need to regress them 80% to the mean. See, now that guy with a 10% on-ice shooting percentage is more likely a 8.4% guy while the 6% shooting percentage player is really a 7.6% guy. See there isn’t really that big of a difference.

Me: But Sidney Crosby almost always has an on-ice shooting percentage of 10+%, not 8.4% and Alex Tanguay’s long-term on-ice shooting percentage is above 10%. Same for Bobby Ryan, Steven Stamkos, Ryan Getzlaf, etc.

Others: Regress. You have to regress. Shooting percentage has very little year over year persistence. On-ice shooting percentage regresses “80% to the mean”.

Me: I get what you are saying but some players clearly do maintain elevated on-ice shooting percentages. Just look at this list and how the players at the top differ from those at the bottom. All first liners at the top and all 3rd line defensive types at the bottom. This doesn’t happen by randomness. It happens due to differences in talent levels.

And then typically the degrades into them giving me some kind of statistical lecture on why one needs to regress despite whatever I observe in actually stats of players. How dare I question statistical methodologies with sound observations.

More recent debates I have had have occurred around a players ability to boost his goalies save percentage (his own on-ice save percentage). Some of you may remember my claims regarding Mike Weavers excellent defensive ability and his ability to boost save percentage. This grew into debates with Eric Tulsky (now of the Carolina Hurricanes) in which I showed some players do seem to systematically improve save percentage while Tulsky downplayed its importance once again turning to the lack of persistence argument.

Recently I have been promoting the idea that players in extreme defensive roles show the greatest ability to boost save percentage with Brandon Sutter being one such player. Shea Weber and Dion Phaneuf as well. However the debate always seems to fall back to the same old argument that without persistence and predictability it isn’t very important. Garret Hohl did this a while back (twice actually) and Matt Cane is the latest to push the idea that without persistence it isn’t important. Matt did try to account for changing roles but in my opinion did so inadequately and I hope to show this in this series of posts.

These save percentage debates are essentially the same as the shooting percentage debates so it is important to point out that those shooting percentage debates eventually had people conceding (to some extent) that driving shot quality and shooting percentage is a talent that matters. For example, in one of Tulsky’s later posts on on-ice shooting percentage he changed his methodology from regressing to a league-wide mean shooting percentage to regressing to a mean shooting percentage of players that get similar amounts of ice time. This meant regressing to something around 9.7% for forwards with the most ice time and something below 6.5% for forwards with the least ice time. The first thing I want to point out is a range of 6.5% to 9.7% is a fairly large range. It means the forwards that get the most ice time will score ~50% more goals on the same number of shots than the players with the least ice time. While there is significant amount of randomness in any single players observed on-ice shooting percentage the talent disparity across all players is actually quite large.

I want to point out is that this is a perfectly valid (and useful) methodology to overcome small sample sizes associated with individual goal statistics. Shooting and save percentages rely on goals scored (its calculated by goals divided by shots) and since goals are relatively infrequent events there is a lot of randomness associated with them. Due to the small sample sizes it is extremely difficult to find persistence in year over year statistics. By grouping players into groups based on similar attributes (i.e. similar ice time) and using aggregate statistics of each group you can overcome small sample size issues. In this case Tulsky regressed to a mean of similarly talented players. Tom Awad grouped players into four tiers in his “What makes good players good” series where he found that shooting percentage  (Awad called this “finishing”) was an important factor in why good players are good. I intend to do similar groupings based on ice time in this series of posts on shooting and save percentages.

(I also take some solace in the fact that people are attempting to develop shot quality models which suggests that people are starting to believe in shot quality even if I don’t think these models are all that useful yet as they still heavily mimic shot metrics.)

Methods

My hypothesis entering this is that players do have an ability to influence their goalies save percentage when they are on the ice just as they have an ability to influence their on-ice shooting percentage. I believe that style of play and roles play a major role in this as well. We know that score effects exist. When a team is leading, and presumably playing defensive hockey, they give up more shots but also see a rise in save percentage. It is my belief that if we are able to identify players that play defensive roles (as opposed to offensive roles) we should see that those players also have a positive impact on their teams save percentage.

In fact, I have actually already done this with relatively good success using two methods of identifying which players play defensive roles. Those two methods were LTIndex and DZone%. In my previous work I defined these as the following:

  1. LTIndex. I haven’t talked about LTIndex much and I don’t yet have it available on Puckalytics.com but it can be used for this purpose. Lets define LeadingTOI% as the percentage of time that his team is defending the lead that he is on the ice for (players TOI in 5v5 leading situations / teams TOI in 5v5 leading situations). Essentially this is what percentage of ice time defending a lead does his coach trust the player to be on the ice for. TrailingTOI% is the same except for in 5v5 trailing situations, or what percentage of ice time the player is given when his team is playing catch up. LTIndex is calculated by taking LeadingTOI% and dividing it by TrailingTOI%. The result is any value greater than 1.00 indicates the player is given a higher percentage of his teams ice time defending a lead than playing catch up. In other words, players with an LTIndex greater than 1.00 are biased towards playing a defensive role while players with an LTIndex less than 1.00 are biased towards playing an offensive role.
  2. DZone%. This is calculated much like OZone% but with DZone faceoffs in the numerator. DZone% = DZ faceoffs / (DZ faceoffs + OZ faceoffs). Players with more defensive zone face offs can likely be considered to be given more defensive roles while players with more offensive zone face offs can be considered to be given more offensive roles. I am using DZone% so that correlations (positive or negative) will mean the same for both DZone% and LTIndex as higher numbers will imply more defensive roles for both stats.

My plan for this project is to actually break things down even further by looking at how six different statistics influence shooting and save percentage. These six stats are:

  1. LeadingTOI% – The percentage of time that his team is defending the lead that he is on the ice for (players TOI in 5v5 leading situations / teams TOI in 5v5 leading situations).
  2. TrailingTOI% – The percentage of time that a players team is trailing that he is on the ice for (players TOI in 5v5 leading situations / teams TOI in 5v5 leading situations).
  3. TeamDZFO% – The percentage of a teams (5v5close situations) defensive zone face offs that the player is on the ice for.
  4. TeamOZFO% – The percentage of a teams (5v5close situations) offensive zone face offs that the player is on the ice for.
  5. 4v5TOI% – The percentage of a teams 4v5 PK ice time that the player is on the ice for.
  6. 5v4TOI% – The percentage of a teams 5v4 PP ice time that the player is on the ice for.

LeadingTOI%, TeamDZFO%, and 4v5TOI% are all indicators of defensive roles and TrailingTOI%, TeamOZFO% and 5v4TOI% are all indicators of offensive roles.

I plan on investigating the relationship between these role indicators and offensive and defensive statistics. Namely a players GF60 Rel, GA60 Rel, CF60 Rel, CA60 Rel, Sh% Rel and Sv% Rel in 5v5close situations. Rel, or relative, statistics attempt to factor out quality of team by taking on-ice minus off-ice statistics. I want to use close situations to minimize impact of score effects as the goal is to identify impact of role above and beyond any impact score effects may have. I could use 5v5tied statistics but the sample sizes start to get really small and 5v5close eliminates most if not all score effects. (There are still valuable uses for “close” data – some people have been too quick to dismiss its value altogether).

Unless otherwise stated I am using two years of data for each player and only including players with at least 600 minutes of 5v5close ice time in those two years.

Here is a preview of what is to come. This chart is for forwards.

PK_vs_PP_vs_GF60

On the x-axis we have 4v5TOI%, a defensive role indicator and on the y-axis we have 5v4TOI%, an offensive role indicator. Throughout these posts I will continue with the defensive role indicator being on the x-axis and the offensive role indicator being on the y-axis. The further to the upper right away from the diagonal 1:1 line the more a players ice time is biased towards offensive roles. The further to the lower left the more a players ice time is biased towards defensive roles. The closer a player is to the upper right the more significant the players role on the team is (i.e. more ice time, first line players). Players closer to the lower left are getting less ice time or face offs (typically 3rd and 4th liners).

The coloured squares are, for this chart, the average GF60Rel for all of the players that fall within that cell. It should not come as a surprise that the players that get the most 5v4 PP ice time also happen to score at the highest rates on the team in 5v5close situations but I’ll leave the more interesting observations for future posts.

Two other things that I should mention. First, for a cell to have a valid average it must have at least four ‘players’ (players being 2-season stats for a single player) within that cell otherwise the cell is left blank. The second thing is that there are some players who get more than 70% of a teams 5v4 ice time. Throughout this process for any players that are ‘off the chart’ I assign them to the closest cell on the chart.

Over the course of the next couple weeks I’ll make a series of posts looking at how roles impact offensive and defensive statistics for forwards and defensemen. My hope is that at the end of this series of posts we will at least be able to acknowledge that players can influence shooting and save percentages even if we can’t (yet) always quantify it with the certainty we really desire.