Roles and Stats Part VIII: Review and Looking Forward

This is my 8th and final post in my series on the relationship between roles and statistics and this will be a review of what we have learned so far and maybe more important what it all means and doesn’t mean.

Series Posts

Review of Findings

In this series of posts I used three different player usage methods for determining whether a player was given an offensive role or a defensive role on his team and then looked at the average statistics of groups of players based on their role. Here are some of the key points we learned.

  1. Forwards in more offensive roles had, generally speaking, better offensive statistics (GF60, CF60, Sh%) than those in defensive roles.
  2. Forwards in offensive roles also had, generally speaking, worse defensive statistics (GA60, Sv%).
  3. While there was a connection between role and GA60 and Sv%, there seemed to be little or no connection between role and CA60.
  4. In general there was a stronger relationship between role and offensive statistics than between role and defensive statistics.
  5. In general, for defensemen there was little or no relationship between their role and their resulting statistics.
  6. Where there was an “in sample” relationship between role and a statistic, that role was also predictive of future performance as well.

Caveats

You may have noticed that I wrote “in general” a lot in the above statements because in every grouping of players there are exceptions to the rule. For example, offense generally comes at the expense of defense however there are some good offensive players that also play good defensively.

We should also not jump to conclusions on the extent that it is the role that is driving the results and not player talent as player talent and abilities is a large part of how roles are assigned. That said, it may very well be the case that players in an offensive role, while more skilled offensively, also take more risks to generate more offensive which results in more and higher quality chances against. This again would be consistent with our observations of score effects.

There may be an eagerness to taking these role definitions and then and then looking at each players performance relative to their role however roles defined by TOI or other usage statistics as I have done here are far from ideal. The depth and makeup of the team along with the coaching style are large factors in a players role and playing style. Furthermore, while using usage statistics to generalize about roles and statistics (as I have done in this series of posts) can give us a better understanding of the game they are not ideal to use in analytics. Essentially TOI and usage statistics are nothing more than a quantification of the coaches opinions on the players. From a fan and informational perspective there is value and legitimate interest in that. From an NHL team organizational perspective that has somewhat less value. Remember, we want to use analytics to drive better decision making, including coaching decisions. If a basis of our analytics starts with “coaches make smart decisions” we aren’t really making an unbiased evaluation are we. I generally believe that coaches do make good decisions the majority of the time however analytics should strive to come to that conclusion independently. Ideally we’d find an independent method of determining role and playing style but that is likely not possible with current limitations in NHL data.

The final thing worth mentioning is that all of these observations are relative to their team. Each team has different talent levels and each team play different styles of play (more aggressive offensive play or more conservative defensive play). It is almost certain that the variance among all players across all teams will be somewhat greater than we have seen in this series of posts.

Looking Forward

You have probably noticed that there is very little “hard math” in this series of posts. By that I mean I have not presented a statistical model or a methodology to take into account how to deal with the fact players can drive and suppress both shooting and save percentage. That is deliberate. It is great to throw a bunch of numbers into a statistical model and see what comes out however it is also great to have a good understanding of the underlying data before doing so.

My intent with this series of posts was to show visually that players can and do impact both shooting and save percentage. Only when we can accept that as fact (and not a random occurrence) will hockey analytics progress forward with better evaluation methods. This will be especially true when we get real player and puck tracking technology implemented in the NHL. If you come from a perspective that players can’t influence save percentage when you get access to player and puck tracking data you will most likely not even consider that when you get your hands the data. For my perspective this is probably the an area where player and puck tracking data is likely to have the most significant impact.

Before we get that data even acknowledging that shot quality exists at both ends of the ice even if it is incredibly difficult to quantify with statistical confidence would be, in my opinion, a large step forward. This isn’t about bashing Corsi. Out shooting your opponent is definitely a good thing, however acknowledging and understanding its limitations makes us all better analysts by forcing ourselves to seek answers in different places (maybe even with traditional scouting).

There is much more that can be done, even with the data we have now. One can look at other ways of identifying roles, quality of competition being a logical next step. One can also seek methods of identifying differences in playing style from team to team possibly through some of the tracking projects that are underway. Are there other factors that we haven’t considered that lead to shot quality? There is more we can do but only when we recognize that not every NHL player plays the same role or has the same objective and this can affect the resulting stats above and beyond player talent.