Jul 232014
 

Tyler Dellow has an interesting post on differences between the Kings and Leafs offensive production. He comes at the problem from a slightly different angle than I have explored in my rush shot series so definitely go give it a read. These two paragraphs discuss a theory of Dellow’s that is interesting.

That’s the sort of thing that can affect a team’s shooting percentage. To take it to an extreme, teams shot 6.2% in the ten seconds after an OZ faceoff win this year; the league average shooting percentage at 5v5 is more like 8%. Of course, when you win an offensive zone draw, you start with the puck but the other team has five guys back and in front of you.

I wonder whether there isn’t something like that going on here that explains LA’s persistent struggles with shooting percentage (as well as those of New Jersey, another team that piles up Corsi but can’t score – solving this problem is one of the burning questions in hockey analytics at the moment). It’s a theory, but one that seems to fit with what Eric’s suggested about how LA generates the bulk of their extra shots. It’s hard for me to explain the Leafs scoring so many more goals in the first 11 seconds after a puck has been carried in, particularly given that I suspect that LA, by virtue of their possession edge, probably enjoyed many more carries into the offensive zone overall.

Earlier today I posted some team rush statistics for the past 7 and past 3 seasons. Let’s look in a little more detail how the Leafs, Kings and Devils performed over the past 3 seasons.

Team RushGF RushSF OtherGF OtherSF RushSh% OtherSh% Rush%
New Jersey 45 540 103 1675 8.33% 6.15% 24.4%
Toronto 66 523 128 1675 12.62% 7.64% 23.8%
Los Angeles 53 609 112 1978 8.70% 5.66% 23.5%

The Leafs scored the most goals on the rush despite the fewest rush shots due to a vastly better shooting percentage (nearly 50% better than the Devils and Kings) on the rush. They do not generate more shots on the rush, but do seem to generate higher quality shots.

The Kings generate by far the most shots in non-rush situations but have the poorest shooting percentage and thus do not score a ton of goals. The Devils don’t generate many non-rush shots and don’t have a great non-rush shooting percentage either and thus posted the fewest goals. The Leafs have had the same number of shots as the Devils but a significantly higher shooting percentage than the Devils and thus scored significantly more non-rush goals.

The Leafs scored 34% of their goals on the rush compared to 32% for the Kings and 30% for the Devils.

Are the Leafs a good rush team? Well, only Boston has scored more 5v5 road rush goals than the Leafs so probably yes but it is mostly because of finishing talent, not shot generating talent. They are 4th last in 5v5 road rush shots.

The Ducks have very similar offense to the Leafs. They don’t get many rush shots but post a really high rush shooting percentage. Anaheim generate a few more non-rush shots than the Leafs but they are very similar offense.

The Kings are a slightly better rush team than the Devils but neither are good and both are weak shooting percentage teams regardless of whether it is a rush or non-rush shot. The Kings make up for this though by generating a lot of shots from offensive zone play where as the Devil’s don’t.

 

Jul 142014
 

I have written a couple of posts (here and here) on rush shots as it relates to shooting percentages and I investigate this further at a later date. First though, I wanted to take a look at save percentages on rush and non-rush shots. Let’s start by looking at teach teams 5v5 road save percentages for the past 7 seasons combined.

RushvsNonRushSavePct_5v5road

A few observations:

  1. Whoa Tampa! That’s a dreadful save percentage on the rush, 2.5% below anyone else. More on this later.
  2. The teams with the best save percentages on the rush are Anaheim, Phoenix, New Jersey, and Boston.
  3. r^2 between rush save % and non-rush save % is just 0.23 which is below what I might have expected.
  4. There is significantly less variability in non-rush save percentage than rush save percentage. The standard deviation in rush save percentage is 1.09% while this standard deviation in non-rush save percentage is 0.43%. All but 7 teams have a non-rush save percentage between 92.2% and 93.0% (a range of 0.8%) while only 21 teams lie in the range from 89.4% to 91.4% (a range of 2.0%). This could be due to greater variation in the smaller sample size of rush shots but the difference in variability is greater than we see with shooting percentage (standard deviation of 0.91% on rush shots and 0.57% on non-rush shots). Could ability to make saves on rush shots be the larger factor in goaltending talent variability?

Which teams give up the highest percentage of “rush” shots? Well, this chart will provide you that answer.

PercentageOfShotsAgainstOnRush_5v5road

Boston and Los Angeles. Who’d have guessed that? Both are teams we would consider good defensive teams and yet a higher percentage of their shots against are of the tougher rush shot variety. Meanwhile Colorado, NY Islanders and Nashville give up the smallest percentage of shots on the rush. Certainly wouldn’t have predicted that for the Islanders and maybe not Colorado either. There doesn’t seem to be any trends that can be extracted from that chart as good teams and bad teams are spread throughout.

Earlier we saw that Tampa had a downright dreadful save percentage on the rush. I wanted to take a look and see if there has been any improvement over the years, particularly last year when they got some good goaltending for the first time since probably Khabibulin.

TampaBaySavePct_Rush_5v5road

For the first time in the past 7 seasons Tampa goaltending provided a league average save percentage on rush shots. Not sure one season is enough to definitively declare their goaltending problems over, but they seem to be on the right path after years of dreadful goaltending.

Since I am a Toronto fan I wanted to take a look at the Leafs as well to see how their goaltending has improved over the years with Reimer and now Bernier.

TorontoSavePct_Rush_5v5road

The Leafs had a good “rush shot” save percentage in 2007-08 but a poor save percentage on other shots with Toskala playing the majority of the games backed up by Raycroft. Everything was bad in 2008-09 though when Toskala once again had the majority of the starts while Curtis Joseph, Martin Gerber and Justin Pogge shared backup duties. Since then things have slowly gotten better, particularly when Reimer came on board the second half of the 2010-11 season and the Leafs have been a better than average team on both rush and non-rush shots the past couple seasons.

I’ll look at some other teams in future posts. If you have any teams that you’d like me to look at (i.e. teams that are particularly interesting due to change in goalies or whatever) let me know and I’ll take a look.

 

Jul 102014
 

Yesterday I introduced the concept of rush shots which are basically any shot we can identify as being a shot taken subsequent to a rush up the ice which can be determined by the location of previous face off, shot, hit, giveaway or takeaway events. If you haven’t read the post from yesterday go give it a read for a more formal definition of what a rush shot is. Today I am going to take a look at how rush shots vary when teams are leading vs trailing as well as investigate home/road differences as arena biases in hits, giveaways and takeaways might have a significant impact on the results.

Leading vs Trailing

One hypothesis I had is that a team defending a lead tends to play more frequently in their own zone and thus have the potential to generate a higher percentage of shots from the rush. Here is a table of leading vs trailing rush shot statistics.

Game Situation Rush Sh% Other Sh% Overall Sh% % Shots on Rush
Leading 10.43% 8.03% 8.62% 24.3%
Trailing 9.36% 7.15% 7.63% 22.0%
Leading-Trailing 1.07% 0.89% 0.98% 2.28%

As expected, teams get a boost in the percentage of overall shots that are rush shots when leading (24.3%) compared to when trailing (2.28%). This higher percentage of shots being rush shots would factor in to the higher shooting percentages but it actually doesn’t seem to be all that significant. The more significant impact still seems to be that teams with the lead experience boosts in shooting percentage on both rush and non-rush shots. The hypothesis that teams have a higher shooting percentage when leading due to the fact that they have more shots on the rush doesn’t seem to be true. It’s just that they shoot better. Note that empty net situations are not considered and thus the shooting percentages when leading are not a result of empty net goals.

 Home vs Road

My concern with home stats is the various arena game recorders dole out hits, giveaways and takeaways at different rates. I determine what is a rush and what isn’t based in part on those events so there is the potential of significant arena biases in rush shot stats. To investigate I looked at the percentage of shots that were rush shots at home and on the road for each team. Here is what I found.

RushShotPercentage_Home_vs_Road

That is about as conclusive as you can get. The rush shot percentage at home is far more variable than on the road with higher highs and lower lows. It is possible that last change line matching usage tactics that coaches can more easily employ at home could account for some of the added variability but my guess is it is mostly due to arena scorer biases. From the chart above I suspect Buffalo, Minnesota, New Jersey  and Pittsburgh don’t hand out hits, giveaways and takeaways as frequently as other arenas. This chart takes a look at last years real time stats (the above chart is for last 7 seasons combined).

HitsGiveawaysTakeaways_Home_vs_Road

Most teams have more hits+giveaways+takeaways on home ice than on the road. The teams that have more on the road than at home are Buffalo, Minnesota, New Jersey, Pittsburgh and St. Louis. Despite comparing a 7-year chart with a 1-year chart the two charts seem to align up fairly well. There does seem to be significant arena biases in rush shot statistics so when looking at team and player stats it is certainly best to consider road stats only.

 

Jul 092014
 

I have been pondering doing this for a while and over the past few days I finally got around to it. I have had a theory for a while that an average shot resulting from a rush up the ice is more difficult than a shot than the average shot that is generated by offensive zone play. It makes sense for numerous reasons:

  1. The rush may be an odd-man rush
  2. The rush comes with speed making it more difficult for defense/goalie to defend.
  3. Shots are probably take from closer in (aside from when a team wants to do a line change rarely do they shoot from the blue line on a rush).

To test this theory I defined a shot off the rush as the following:

  • A shot within 10 seconds of a shot attempt by the other team on the other net.
  • A shot within 10 seconds of a face off at the other end or in the neutral zone.
  • A shot within 10 seconds of a hit, giveaway or takeaway in the other end or the neutral zone.

I initially looked at just the first two but the results were inconclusive because the number of rush events were simply too small so I added giveaway/takeaway and hits to the equation and this dramatically increased the sample size of rush shots. This unfortunately introduces some arena bias into the equation as it is well known that hits, giveaways and takeaways vary significantly from arena to arena. We will have to keep this in mind in future analysis of the data and possibly consider just road stats.

For now though I am going to look at all 5v5 data. Here is a chart of how each team looked in terms of rush and non-rush shooting percentages.

Rush_vs_NonRush_ShootingPct_2007-14b

So, it is nice to see that the hypothesis holds true. Every team had a significantly higher shooting percentage on “rush” shots than on shots we couldn’t conclusively define as a rush shot (note that some of these could still be rush shots but we didn’t have an event occur at the other end or neutral zone to be able to identify it as such). As a whole, the league has a rush shot shooting percentage of 9.56% over the past 7 seasons while the shooting percentage is just 7.34% on shots we cannot conclusively define as a rush shot. Over the 7 years 23.5% of all shots were identified as rush shots while 28.6% of all goals scored were on the rush.

In future posts over the course of the summer I’ll investigate rush shots further including but not limited to the following:

  • How much does the frequency of rush shots drive a teams/players overall shooting/save percentages?
  • Are score effects on shooting/save percentages largely due to increase/decrease in rush shot frequency?
  • Are there teams/players that are better at reducing number of rush shots?
  • Can rush shots be used to identify and quantify “shot quality” in any useful way?
  • How does this align with the zone entry research that is being done?

 

 

Jun 232014
 

More often than not the first thing I look at when I want to evaluate a player is their WOWY stats to see if the player boosts the performance of their teammates or suppress it when he is on the ice. Let’s take a look at a WOWY comparision of Umberger and Hartnell starting with some links to their WOWY pages.

When on any of those pages you can click “Visualize this table” to get some charts that I find are often a quick way of getting an overview of the player in question. For example, here is a CF% WOWY chart for Hartnell from last year.

Hartnell-CF-WOWY-2013-14

In these charts it is better to have bubbles below and to the right of the one-to-one diagonal line from that runs from the lower left to the upper right. For Hartnell in 2013-14 every single teammate was the the lower right of this diagonal line which is really good. Not a lot of players have charts this nice. If you go back and look at previous years you will see that Hartnell has accomplished this relatively consistently. This is a good thing. Now let’s take a look at Umberger’s.

Umberger-CF-WOWY-2013-14

That is a much less impressive chart as the majority of Umberger’s team mates have performed better when not playing with him. This is not good and yet is is fairly typical for Umberger to have WOWY charts that look like this.

This is a table of how Umberger’s line mates performed with and without Umberger last season. Listed are all forwards who played at least 100 minutes of 5v5 ice time with Umberger.

Line mate With Umberger Without Umberger
Ryan Johansen 50.2% 50.8%
Nick Foligno 50.4% 52.0%
Artem Anisimov 40.1% 53.3%
Blake Comeau 46.1% 54.6%
Mark Letestu 42.8% 52.1%

And now for Hartnell’s line mates who played at least 100 minutes with Hartnell last year.

Line mate With Hartnell Without Hartnell
Claude Giroux 55.7% 49.5%
Jakub Voracek 57.1% 52.5%
Brayden Schenn 51.9% 46.3%
Wayne Simmonds 53.9% 46.3%

Again, you can go back to previous seasons and the general trend for the two players is pretty much the same. Players perform worse when playing with Umberger than when not and players perform better when playing with Hartnell than when not.

From a WOWY perspective, Umberger is a below average player and Hartnell is an above average player. In fact there aren’t many players that have WOWY charts that look better than Hartnell’s except for the true star players (such as Toews, or Bergeron, or Kopitar, etc.).  Hartnell in my opinion is easily a top 6 player. Umberger I am not sure I’d really want on my team in any significant role. With this trade the Blue Jackets get better in two ways. First by adding a good player in Hartnell and second by subtracting a poor player in Umberger (classic case of addition by subtraction).

 

May 112014
 

I often feel that I am the sole defender of goal based hockey analyitics in a world dominated by shot attempt (corsi) based analytics. In recent weeks I have often heard the pro-corsi crowd cite example after example of where corsi-based analytics “got it right” or “predicted something fairly well”. While it is always good to be able to cite examples where you got things right a fair an honest evaluation looks at the complete picture, not just the good outcomes. Otherwise it is analytics by anecdotes which is an oxymoron if there every was one.

For example, Kent Wilson of FlamesNation.ca recent wrote about the “Dawning of the Age of Fancy Stats” in which he cited several instances of where hockey analytics got it right or did well in predicting outcomes.

The big test case which seems to have moved the needle in favour of the nerds is, of course, the Toronto Maple Leafs. Toronto came into the season with inflated expectations after an outburst of percentages during the lock-out shortened year saw them break into the post-season. Their awful underlying numbers caused the stats oriented amongst us to be far more circumspect about their chances, of course.

Toronto is the recent example that the hockey analytics crowd likes to bring up in support of their case but it is just one example. We don’t hear much about how many predicted the Ottawa Senators to be in the playoffs and some even had them challenging for the top spot in the eastern conference. We don’t hear much about how the New Jersey Devils missed the playoffs yet again despite having the 5th best 5v5close Fenwick% in the league, the year after missing the playoffs with the 3rd best 5v5close Fenwick% in the league. If we are truly interested in hockey analytics we need a complete and unbiased assessment of all outcomes, not just the ones that support our underlying belief.

In the same article Kent Wilson quoted a tweet from Dimitri Filipovic about the success of Corsi in predicting outcomes of playoff series.

Relevant #fact: since ’08 playoffs, teams that were 5+ % better than their opponent in 5v5 fenwick close during the regular season are 25-7.

While interesting, what it really doesn’t tell us a whole lot more than “when one team is significantly better at outshooting their opponents they more often than not win”. Well, that really isn’t saying a whole lot. It is more or less saying, when a dominant team plays a mediocre team, the dominant team usually wins. Not really that interesting when you think of it that way.

Here is another fact that puts that into perspective. Since the 2008 playoffs, the team with the better 5v5close Fenwick% has a 53-35-2 record (there were 2 cases where teams had identical fenwick% to 1 decimal place). That actually makes it sound like 5v5close Fenwick% is predictive overall, not just in cases where one team is significantly better than another. Of course, if we look at goals we find that the team with the better 5v5close goal% has a 54-34-1 record. In other words, 5v5close possession stats did no better at predicting playoff outcomes than 5v5close goal stats. It is easy to throw out stats that support a point of view, but it is far more important to look at the complete picture. That is what analytics is about.

A similar statistic was promoted by Michael Parkatti in a recent talk on hockey analytics at the University of Alberta. In that talk Parkatti stated that of the last 15 Stanley Cup winners all but 3 had a “ShotShare” (all situations) of at least 53%. The exceptions were Pittsburgh in 2009, Boston in 2011 and Carolina in 2006. I will note that it appears that all three of these teams are below 51% and 2009 Penguins were below 50%. That seems sort of impressive but I did some digging myself and found that every Stanley Cup winner since 1980 had a “GoalShare” (all situations) greater than 52%. Every single one. No exceptions. I didn’t look at any cup winners pre-1980 but the trend may very well go back a lot further. As impressive as 12 of 15 is, 34 of 34 is far more impressive.

Here is the thing. We know that goal percentage correlates with winning far better than corsi percentage. This is an indisputable fact. It is actually quite a bit better. The sole reason we use corsi is that goals are infrequent events and thus not necessarily indicative of true talent due to small sample size issues. This is a fair argument and one that I accept. In situations where you have small sample sizes definitely use corsi as your predictive metric (but understand its limitations). The question that needs to be answered is what constitutes a small sample size and more importantly what sample size do we need such that goals become as good or better of a predictor of future events than corsi. I have pegged this crossing point at about 1 seasons worth of data, maybe a bit more if looking at individual players who may not be getting 20 minutes of ice time a game (my guess is around >750 minutes of ice time is where I’d start to get more comfortable using goal data than corsi data). I am certain not everyone agrees but I haven’t see a lot of analyses attempting to find this “crossing point”.

Let’s take another look at how well 5v5close Fenwick% and Goal% predict playoff outcomes again but lets look by season rather than overall.

FF% GF%
2008 7-7-1 6-9
2009 9-6 11-4
2010 9-6 11-4
2011 10-5 11-4
2012 7-7-1 7-7-1
2013 11-4 8-7
Total 53-35-2 54-35-1

In full seasons not affected by lockouts we find that GF% was generally the better predictor (only 2008 did GF% under perform FF%) but in last years lockout shortened season FF% significantly outperformed GF%. Was this a coincidence or is it evidence that 48 games is not a large enough sample size to rely on GF% more than CF% but 82 games probably is?

I have seen numerous other examples in recent weeks where “analytics” supporters have used what amounts to not much more than anecdotal evidence to support their claims. This is not analytics. Analytics is a fair, unbiased and complete fact based assessment of reality. Showing why a technique is a good predictor some of the time is not enough. You need to show why it is overall a better predictor all of the time or at least define when it is and when it isn’t.

I recently wrote an article on whether last years statistics predicted this years playoff teams and found that GF% seemed to do at least as well as CF% despite last season being a lock-out shortened year.

With all that said, you will frequently find me using “possession” statistics so I certainly don’t think they are useless. It is just my opinion that puck possession is just one aspect of the game and puck possession analytics has largely been oversold when it comes to how useful it is as a predictor. Conversely goal based analytics has been largely given a bad rap which I find a little unfortunate.

(Another article worth reading is Matt Rudnitsky’s MONEYPUCK: Why Most People Need To Shut Up About ‘Advanced Stats’ In The NHL.)

 

Sep 142013
 

A while back I came up with a stat which at the time I called LT Index which is essentially the percentage of a players teams ice time when leading that the player is on the ice for divided by the percentage of a players teams ice time when trailing that the player is on the ice for (in 5v5 situations and only in games in which the player played). LT Index standing for Leading-Trailing Index. I have decided to rename this statistic to Usage Ratio since it gives us an indication of whether players are used more in defensive situations (i.e. leading and protecting a lead and thus a Usage Ratio above 1.00) or in offensive situations (i.e. when trailing and in need of a goal and thus a Usage Ratio less than 1.00). I think it does a pretty good job of identifying how a player is used.

I then compared players Usage Index to their 5v5 tied statistics using the theory that a player being used in a defensive role when leading/trailing is more likely to be used in a defensive role when the game is tied. This is also an out of sample comparison (which is always a nice thing to be able to do) since we are using leading/trailing situations to identify offensive vs defensive players and then comparing to 5v5 tied situations that in no way overlap the leading or trailing data.

Let’s start by looking at forwards using data over the last 3 seasons and including all forwards with >500 minutes of 5v5 tied ice time. The following charts compare Usage Ratio with 5v5 Tied CF%, CF60 and CA60.

UsageRatiovsCFPct

UsageRatiovsCF60

UsageRatiovsCA60

Usage Ratio is on the horizontal axis with more defensive players to the right and offensive players to the left.

Usage Ratio has some correlation with CF% but that correlation is solely due to it’s connection with generating shot attempts for and not for restricting shot attempts against. Players we identify as offensive players via the Usage Ratio statistic do in fact generate more shots but players we identify as defensive players do not suppress opposition shots any. In fact, Usage Ratio and 5v5 tied CA60 is as uncorrelated as you can possibly get. One may attempt to say this is because those defensive players are playing against offensive players (i.e. tough QoC) and that is why but if this were the case then those offensive players would be playing against defensive players (i.e. tough defensive QoC) and thus should see their shot attempts suppressed as well. We don’t observe that though. It just seems that players used as defensive players are no better at suppressing shot attempts against than offensive players but are, as expected, worse at generating shot attempts for.

Before we move on to defensemen let’s take a look at how Usage Ratio compares with shooting percentage and GF60.

UsageRatiovsShPct

 

UsageRatiovsGF60

As seen with CF60, Usage Ratio is correlated with both shooting percentage and GF60 and the correlation with GF60 is stronger than with CF60. Note that the sample size for 3 seasons (or 2 1/2 actually) of 5v5 tied data is about the same as the sample size for one season of 5v5 data (players in this study have between 500 and 1300 5v5 tied minutes which is roughly equivalent of how many 5v5 minutes forwards play over the course of one full season).

FYI, the dot up at the top with the GF60 above 5 is Sidney Crosby (yeah, he is in a league of his own offensively) and the dot to the far right (heavy defensive usage) is Adam Hall.

Now let’s take a look at defensemen.

UsageRatiovsCFPctDefensemen

UsageRatiovsCF60Defensemen

UsageRatiovsCA60Defensemen

There really isn’t much going on here and how a defenseman is used really does’t tell us much at all about their 5v5 stats (only marginal correlation to CF60). As with forwards, defensemen that we identify as being used in a defensive are not any better at reducing shots against than defensemen we identify as being used in an offensive manner.

To summarize the above, players who get more minutes when playing catch up are in fact better offensive players, particularly when looking at forwards but players who get more minutes when protecting a lead are not necessarily any better defensively. We do know that there are better defensive players (the range of CA60 among forwards is similar to the range of CF60 so if there is offensive talent there is likely defensive talent too), and yet coaches aren’t playing these defensive players when protecting a lead. Coaches in general just don’t know who their good defensive players are.

Still not sold on this? Well, let’s compare 5v5 defensive zone start percentage (percentage of face offs taken in the defensive zone) to CF60 and CA60 (for forwards) in 5v5 tied situations.

DefensiveFOPctvsCF60

Percentage of face offs in the defensive zone is on the horizontal axis and CF60 is on the vertical axis. This chart is telling us that the fewer defensive zone face offs a forward gets, and thus likely more offensive face offs, the more shot attempts for they produce. In short, players who get offensive zone starts get more shot attempts.

DefensiveFOPctvsCA60

The opposite is not true though. Players who get more defensive face offs don’t give up any more or less shots than their low defensive zone face off counterparts. This tells me that if there is any connection between zone starts and CF% it is solely due to the fact that players who get offensive zone starts are better offensive players and not because players who get defensive zone starts are better defensive players.

You might again be saying to yourself ‘the players who are getting the defensive zone starts they are playing against better offensive players so doesn’t make sense that their CA60 is inflated above their talent levels (which presumably is better than average defensively)?  This might be true, but if zone starts significantly impacted performance (as would be the case if that last statement were true), either directly or indirectly because zone starts are linked to QoC, then there should be more symmetry between the charts. There isn’t though. Let’s look at what these two charts tell us:

  1. The first chart tells us that players who get offensive zone starts generate more shot attempts.
  2. The second chart tells us that players who get defensive zone starts don’t give up more shots attempts against.

If zone starts were a major factor in results, those two statements don’t jive. How can one side of the ledger show an advantage and the other side of the ledger be neutral? The way those statements can work in conjunction with each other is if zone starts don’t significantly impact results which is what I believe (and have observed before).

But, if zone starts do not significantly impact results, then the results we see in the two charts above are driven by the players talent levels. Knowing that we once again can observe that coaches are doing a decent job of identifying offensive players to start in the offensive zone but are doing a poor job at identifying defensive players to play in the defensive zone.

All of this is to say, NHL coaches generally do a poor job at identifying their best defensive players so if you think that guy who is getting all those defensive zone starts (aka ‘tough minutes’) are more likely to be defensive wizards, think again. They may not be.

 

Aug 022013
 

In Rob Vollman’s Hockey Abstract book he talks about the persistence and its importance when it comes to a particular statistics having value in hockey analytics.

For something to qualify as the key to winning, two things are required: (1) a close statistical correlation with winning percentage and (2) statistical persistence from one season to another.

More generally, persistence is a prerequisite for being able to call something a talent or a skill and how close it correlates with winning or some other positive outcome (such as scoring goals) tells us how much value that skill has.

Let’s look at persistence first. The easiest way to measure persistence is to look at the correlation of that statistics over some chunk of time vs some future chunk of time. For example, how well does a stat from last season correlate with the same stat this season (i.e. year over year correlation). For some statistics such as shooting percentages it may even be necessary to go with even larger sample sizes such as 3 year shooting percentage vs future 3 year shooting percentages.

One mistake that many people make when doing this is conclude that the lack of correlation and thus lack of persistence means that the statistics is not a repeatable skill and thus, essentially, random. The thing is, the method for how we measure persistence can be a major factor in how well we can measure persistence and how well we can measure true randomness. Let’s take two methods for measuring persistence:

  1.  Three year vs three year correlation, or more precisely the correlation between 2007-10 and 2010-13.
  2.  Even vs odd seconds over the course of 6 seasons, or the statistic during every even second vs the statistic during every odd second.

Both methods split the data roughly in half so we are doing a half the data vs half the data comparison and I am going to do this for offensive statistics for forwards with at least 1000 minutes of 5v5 ice time in each half. I am using 6 years of data so we get large sample sizes for shooting percentage calculations. Here are the correlations we get.

Comparison 0710 vs 1013 Even vs Odd Difference
GF20 vs GF20 0.61 0.89 0.28
FF20 vs FF20 0.62 0.97 0.35
FSh% vs FSh% 0.51 0.73 0.22

GF20 is Goals for per 20 minutes of ice time. FF20 is fenwick for (shots + missed shots) per 20 minutes of ice time. FSh% is Fenwick Shooting Percentage or goals/fenwick.

We can see that the level of persistence we identify is much greater when looking at even vs odd minute correlation than when looking at 3 year vs 3 year correlation. A different test of persistence gives us significantly different results. The reason for this is that there are a lot of other factors that come into play when looking at 3 year vs 3 year correlations than even vs odd correlations. In the even vs odd correlations factors such as quality of team mates, quality of competition, zone starts, coaching tactics, etc. are non-factors because they should be almost exactly the same in the even minutes as the odd minutes. This is not true for the 3 year vs 3 year correlation. The difference between the two methods is roughly the amount of the correlation that can be attributed to those other factors. True randomness, and thus true lack of persistence, is essentially the difference between 1.00 and the even vs odd correlation. This equates to 0.11 for GF20, 0.03 for FF20 and 0.27 for FSh%.

Now, lets look at how well they correlate with a positive outcome, scoring goals. But instead of just looking at that lets combine it with persistence by looking at how well predict ‘other half’ goal scoring.

Comparison 0710 vs 1013 Even vs Odd Difference
FF20 vs GF20 0.54 0.86 0.33
GF20 vs FF20 0.44 0.86 0.42
FSh% vs GF20 0.48 0.76 0.28
GF20 vs FSh% 0.57 0.77 0.20

As you can see, both FF20 and FSh% are very highly correlated with GF20 but this is far more evident when looking at even vs odd than when looking at 3 year vs 3 year correlations. FF20 is more predictive of ‘other half’ GF20 but not significantly so but this is likely solely due to the greater randomness of FSh% (due to sample size constraints) since FSh% is more correlated with GF20 than FF20 is. The correlation between even FF20 and even GF20 is 0.75 while the correlation between even FSh% and even GF20 is 0.90.

What is also interesting to note is that even vs odd provides greater benefit for identifying FF20 value and persistence than for FSh%. What this tells us is that the skills related to FF20 are not as persistent over time as the skills related to FSh%. I have seen this before. I think what this means is that GMs are valuing shooting percentage players more than fenwick players and thus are more likely to maintain a core of shooting percentage players on their team while letting fenwick players walk. Eric T. found that teams reward players for high shooting percentage more than high corsi so this is likely the reason we are seeing this.

Now, let’s take a look at how well FF20 correlates with FSh%.

Comparison 0710 vs 1013 Even vs Odd Difference
FF20 vs FSh% 0.38 0.66 0.28
FSh% vs FF20 0.22 0.63 0.42

It is interesting to note that fenwick rates are highly correlated with shooting percentages especially when looking at the even vs odd data. What this tells us is that the skills that a player needs to generate a lot of scoring chances are a similar set of skills required to generate high quality scoring chances. Skills like good passing, puck control, quickness can lead to better puck possession and thus more shots but those same skills can also result in scoring at a higher rate on those chances. We know that this isn’t true for all players (see Scott Gomez) but generally speaking players that are good at controlling the puck are good at putting the puck in the net too.

Finally, let’s look at one more set of correlations. When looking at the the above correlations for players with >1000 minutes in each ‘half’ of the data there are a lot of players that have significantly more than 1000 minutes and thus their ‘stats’ are more reliable. In any given year a top line forward will get 1000+ minutes of 5v5 ice time (there were 125 such players in 2011-12) but generally less than 1300 minutes (only 5 players had more than 1300 minutes in 2010-11). So, I took all the players that had more than 1000 even and odd minutes over the course of the past 6 seasons but only those that had fewer than 2600 minutes in total. In essense, I took all the players that have between 1000 and 1300 even and odd minutes over the past 6 seasons. From this group of forwards I calculated the same correlations as above and the results should tell us approximately how reliable (predictive) one seasons worth of data is for a front line forward assuming they played in exactly the same situation the following season.

Comparison Even vs odd
GF20 vs GF20 0.82
FF20 vs FF20 0.93
FSh% vs FSh% 0.63
FF20 vs GF20 0.74
GF20 vs FF20 0.77
FSh% vs GF20 0.65
GF20 vs FSh% 0.66
FF20 vs FSh% 0.45
FSh% vs FF20 0.40

It should be noted that because of the way in which I selected the players (limited ice time over past 6 seasons) to be included in this calculation there is an abundance of 3rd liners with a few players that reached retirement (i.e. Sundin) and young players (i.e. Henrique, Landenskog) mixed in. It would have been better to take the first 2600 minutes of each player and do even/odd on that but I am too lazy to try and calculate that data so the above is the best we have. There is far less diversity in the list of players used than the NHL in general so it is likely that for any particular player with between 1000 and 1300 minutes of ice time the correlations are stronger.

So, what does the above tell us? Once you factor out year over year changes in QoT, QoC, zone starts, coaching tactics, etc.  GF20, FF20 and FSh% are all pretty highly persistent with just one years worth of data for a top line player. I think this is far more persistent, especially for FSh%, than most assume. The challenge is being able to isolate and properly account for changes in QoT, QoC, zone starts, coaching tactics, etc. This, in my opinion, is where the greatest challenge in hockey analytics lies. We need better methods for isolating individual contribution, adjusting for QoT, QoC, usage, etc. Whether that comes from better statistics or better analytical techniques or some combination of the two only time will tell but in theory at least there should be a lot more reliable information within a single years worth of data than we are currently able to make use of.

 

Jun 182013
 

If you have been following the discussion between Eric T and I you will know that there has been a rigorous discussion/debate over where hockey analytics is at, where it is going, the benefits of applying “regression to the mean” to shooting percentages when evaluating players. For those who haven’t and want to read the whole debate you can start here, then read this, followed by this and then this.

The original reason for my first post on the subject is that I rejected Eric T’s notion that we should “steer” people researching hockey analytics towards “modern hockey thought” in essence because I don’t we should ever be closed minded, especially when hockey analytics is pretty new and there is still a lot to learn. This then spread into a discussion of the benefits of regressing shooting percentages to the mean, which Eric T supported wholeheartedly while I suggested that I think further research into isolating individual talent even goal talent through adjusting for QoT, QoC, usage, score effects,  coaching styles, etc. can be equally beneficial and focus need not be on regressing to the mean.

In Eric T’s last post on the subject he finally got around to actually implementing a regression methodology (though he didn’t post any player specifics so we can’t see where it is still failing miserably) in which he utilized time on ice to choose a mean for which a players shooting percentage should regress to. This is certainly be better than regressing to the league-wide mean which he initially proposed but the benefits are still somewhat modest. The results for players who played 1000 minutes in the 3 years of 2007-10 and 1000 minutes in the 3 years from 2010-13 showed the predictive power of his regressed GF20 to predict future GF20 was 0.66 which was 0.05 higher than the 0.61 predictive power raw GF20. So essentially his regression algorithm improved predictive power by 0.05 while there still remains 0.34 which is unexplained. The question I attempt to answer today is for a player who has played 1000 minutes of ice time, what is the amount of his observed stats that is true randomness and what amount is simply unaccounted for skill/situational variance.

When we look at 2007-10 GF20 and compare it to 2010-13 GF20 there are a lot of factors that can explain the differences from a change in quality of competition, a change in quality of team mates, a change in coaching style, natural career progression of the player, zone start usage, and possibly any number of other factors that might come into play that we do not currently know about as well as true randomness. To overcome all of these non-random factors that we do not yet know how to fully adjust for in order to get a true measure of the random component of a players stats we need to be able to get two sets of data that have attributes (QoT, QoC, usage, etc) as similar to each other as possible. The way I did this was to take each of the 6870 games that have been played over the past 6 seasons and split them into even and odd games and calculate each players GF20 over each of those segments. This should, more or less, split a players 6 years evenly in half such that all those other factors are more or less equivalent across halves. The following table shows how predicting the even half is at predicting the odd half based on how many total minutes (across both halves) that the player has played.

Total Minutes GF20 vs GF20
>500 0.79
>1000 0.85
>1500 0.88
>2000 0.89
>2500 0.88
>3000 0.88
>4000 0.89
>5000 0.89

For the group of players with more than 500 minutes of ice time (~250 minutes or more in each odd/even half) the upper bound on true randomness is 0.21 while the predictive power of GF20 is 0.79. With greater than 1000 minutes randomness drops to 0.15 and with greater than 1500 minutes and above the randomness is around 0.11-0.12. It’s interesting that setting the minimum above 1500 minutes (~750 in each even/odd half) of data doesn’t necessarily reduce the true randomness in GF20 which seems a little counter intuitive.

Let’s take a look at the predictive power of fenwick shooting percentage in even games to predict fenwick shooting percentage in odd games.

Total Minutes FSh% vs FSh%
>500 0.54
>1000 0.64
>1500 0.71
>2000 0.73
>2500 0.72
>3000 0.73
>4000 0.72
>5000 0.72

Like GF20, the true randomness of fenwick shooting percentage seems to bottom out at 1500 minutes of ice time and there appears to be no benefit to going with increasing the minimum minutes played.

To summarize what we have learned we have the following which is for forwards with >1000 minutes in each of 2007-10 and 2010-13.

GF20 predictive power 3yr vs 3yr 0.61
True Randomness Estimate 0.11
Unaccounted for factors estimate 0.28
Eric T’s regression benefit 0.05

There is no denying that a regression algorithm can provide modest improvements but this is only addressing 30% of what GF20 is failing to predict and it is highly doubtful that efforts to improve the regression algorithm any more will result in anything more than marginal benefits. The real benefit will come from researching the other 70% we don’t know about. It is a much more difficult  question to answer but the benefit could be far more significant than any regression technique.

Addendum: After doing the above I thought, why not take this all the way and instead of doing even and odd games do even and odd seconds so what happens one second goes in one bin and what happens the following second goes in the other bin. This should absolutely eliminate any differences in QoC, QoT, zone starts, score effects, etc. As you might expect, not a lot has changed but the predictive power of GF20 increases marginally, particularly when dealing with lower minute cutoffs.

Total Minutes GF20 vs GF20 FSh% vs FSh%
>500 0.81 0.58
>1000 0.86 0.68
>1500 0.88 0.71
>2000 0.89 0.73
>2500 0.89 0.73
>3000 0.90 0.75
>4000 0.90 0.73
>5000 0.89 0.71

 

May 152013
 

After last weeks untimely pinch by Dion Phaneuf in game 4 that led to an overtime goal and the Bruins taking a 3-1 lead in the first round series there was a lot of evaluation of Phaneuf as a defenseman both good and bad. I was intending to write an article to discuss the relative merits of Dion Phaneuf and attempt to get an idea of where he stands among NHL defensemen but in the process of researching that I came across some interesting Phaneuf stats that I think deserve their own post so here it is.

My observation was with respect to Phaneuf’s usage and performance when the Leafs are leading and when they are trailing over the previous 3 seasons. Let’s start of by looking at Phaneuf’s situational statistics over the past 3 seasons.

5v5 5v5close 5v5tied Leading Trailing
G/60 0.222 0.175 0.101 0.156 0.408
Pts/60 0.700 0.670 0.660 0.420 1.020
IPP 30.1% 31.1% 34.2% 20.0% 34.5%
GF20 0.773 0.721 0.640 0.692 0.986
GA20 0.841 0.760 0.943 0.865 0.714
GF% 47.9% 48.7% 40.4% 44.4% 58.0%
CF20 18.316 18.113 18.159 15.195 21.542
CA20 20.686 21.418 21.880 22.982 17.223
CF% 47.0% 45.8% 45.4% 39.8% 55.6%
OZ% 28.0% 26.7% 25.2% 24.2% 34.5%
DZ% 31.8% 30.3% 29.7% 37.5% 28.5%
NZ% 40.3% 43.0% 45.0% 38.3% 37.0%
DZBias 103.9 103.6 104.4 113.3 94.0
TeamDZBias 108.9 109 107 115.2 100.8
DZBiasDiff -5 -5.4 -2.6 -1.9 -6.8

Most of the stats above the regular readers should be familiar with but if you are not you can reference my glossary here. The one stat that I have not used before is DZBias. DZBias is defined as 2*DZ% + NZ% and thus anything over 100 indicates the player has a bias towards starting shifts in the defensive zone and anything under 100 the player has a bias towards starting in the offensive zone. I prefer this to OZone% which is OZStarts/(OZStarts+DZStarts) because it takes into account neutral zone starts as well. TeamDZBias is the zone start bias of the Leafs over the past 3 seasons and DZBiasDiff is Phaneuf’s DZBias minus the teams DZBias and provides a zone start bias relative to the team. Anything less than 0 indicates usage is more in the offensive zone relative to his teammates.

So, what does this tell us about Phaneuf.  Well, there isn’t a huge variation in either the zone start usage or the results during 5v5, 5v5close and 5v5tied situations so the focus should be on the differences between 5v5leading and 5v5trailing which are significant.

Typical score effects are when leading a team gives up more shots but of lower quality (defensive shells protect the danger zone in front of the net but allow more shots from the perimeter) and takes fewer shots but of higher quality (probably a result of more odd-man rushes due to pinching defensemen of the trailing team).  Phaneuf seems to take this concept to the extreme but more importantly Phaneuf seems to excel best in an offensive role and struggles in a defensive role. When the Leafs are trailing Phaneuf has  0.408G/60 (10th of 180 defensemen) and 1.02 points/60 (36th of 180 defensemen) but when leading Phaneuf falls to 0.156 G/50 (64th of 177 defensemen) and 0.42 points/60 (137th of 177 defensemen). Furthermore, Phaneuf’s involvement in the offensive zone drops off significantly when leading (IPP drops from 34.5% when trailing to 20.0% when leading).

In terms of on-ice stats, Phaneuf’s CF% drops from 55.6% when trailing (79th of 180 defensemen) to a very poor 39.8% when leading (164th of 177 defensemen).  Some may be thinking this is due to zone starts but Phaneuf is getting above average offensive zone starts both when trailing (ranks 100th of 180 defensemen) and when leading (ranks 154th of 177) and using even the most aggressive zone start adjustments in no way will account for the difference. Similar observations can be made with on-ice goal stats as well. Let’s look at how Phaneuf ranks among defensemen over the past 3 seasons.

Leading (of177) Trailing ( of 180)
GF20 109 25
GA20 125 71
GF% 126 36
CF20 128 31
CA20 174 154
CF% 164 79

That is a pretty significant improvement in rankings when trailing over when leading, especially in the offensive statistics (GF20, CF20). If zone starts aren’t a factor, might line mates be? He are Phaneuf’s most frequent defense partners:

Trailing:  Gunnarsson (364:33, 31.0%), Beauchemin(212:07, 18,0%), Aulie(162:09, 13.8%)

Leading: Gunnarsson (376:16, 32.5%), Aulie(234:17, 20.3%), Beauchemin(166:30, 14.4%)

Playing more with Beauchemin and less with Aulie when trailing ought to help, particularly ones offensive stats, but I doubt that is going to account for that much of a difference. Also, when leading Phaneuf has a 41.2CF% with Gunnarsson and when trailing that spikes to 54.6%. When leading Phaneuf and Beauchemin have a CF% of 37.3% and when trailing that spikes to 57.7%. With Aulie the difference is 36.6% vs 49.3%. Regardless of which defense partner Phaneuf is with, their stats dramatically improve when playing in catch up situation than when in trailing situations.

The same is true for forwards. When protecting a lead Phaneuf plays more with Grabovski and Kulemin but when playing catch up he plays a bit more with Kessel and Bozak but for all of those forwards Phaneuf’s numbers with them are hugely better when playing catch up than when protecting a lead and playing with Grabovski and Kulemin more when playing with a lead should only help his statistics as they are generally considered the Leafs better corsi players.

Let’s take a look at a chart of Phaneuf’s corsi WOWY’s when leading and when trailing.

Leading:

PhaneufLeadingCorsiWOWY201013

As you can see, when leading the majority of Phaneuf’s team mates are to the left of the diagonal line which means they have a better corsi% without Phaneuf than with.

Trailing:

PhaneufTrailingCorsiWOWY201013

When trailing the majority of Phaneuf’s team mates are near or to the right of the diagonal line which means they generally have better corsi% statistics when with Phaneuf than when apart.

So the question arises, why is this? It doesn’t seem to be zone starts. It doesn’t seem to be changes in line mates and it isn’t that the team as a whole automatically becomes a great corsi% team when trailing which Phaneuf could benefit from. When leading Phaneuf’s corsi% is 39.8% which is worse than the teams 41.2% and when trailing Phaneuf’s corsi% is 55.6% which is better than the teams 54.4%. It seems to me that the conclusion we must draw from this is that Phaneuf has been poor at protecting a lead relative to his team mates and we know his team mates have been poor at protecting a lead. Where Phaneuf excels is when he is asked to engage offensively be that when playing catch up hockey or when playing on the PP (Phaneuf’s PP statistics are pretty solid). From the first chart we know that Phaneuf has a slight bias towards more offensive zone starts (relative to his team mates) and when we dig into the numbers further it probably shows that he should be given even more offensive opportunities and given fewer defensive ones because he seems like a much better player when asked to be engaged offensively than when he is asked to be a shut down defenseman.

Acquiring a quality shut down defenseman (ideally two) this off season must be the #1 priority of Maple Leaf management and Phaneuf’s usage must shift further away from multi-purpose heavy work load defenseman to primarily an offensive usage defenseman.