Feb 052012
 

One of my beefs in the analysis and evaluation of hockey players is the notion that PDO (on-ice shooting percentage plus on-ice save percentage) can be used as a proxy for luck.  A perfect example of how PDO is used as a proxy for luck is this article by Neil Greenberg about the Washington Capitals.

For example, when Alex Ovechkin has been on the ice during even strength this season, the team has a shooting percentage of 8.2 percent and has saved shots at a rate of .917. So that makes his PDO value 999 (.082+.917=.999), which is almost exactly the league average. In other words, Ovechkin has seen neither very good nor very bad “puck luck” this season.

What’s useful about this metric is that it’s “unstable,” and over a large-enough sample will regress to 1000. Why 1000? Because every shot that is a goal is a shot not saved, and vice versa.

My beef with such an analysis is the notion that for all players PDO regresses to 1000 and any players with PDO above 1000 are lucky  and any players with a PDO below 1000 are unlucky.  While I do believe luck can influence PDO over small sample sizes, not all players have a natural PDO level of 1000 and there are two reasons why.

1.  Not all players play in front of perfectly average goalies which will have a major impact on the save percentage portion of PDO.

2. Players can drive shooting percentages.

To show you what I mean on point 2, I took 4 years (2007-08 to 2010-11) of 5v5 zone start adjusted data and grouped forwards based on their ice time over those 4 years and then calculated the on-ice shooting and save percentages and PDO for each group.  Here is what I found.

TOI (minutes) SH% SV% PDO
<500 7.5% 90.9% 983.5
500-999 7.9% 91.2% 991.2
1000-1499 8.0% 91.2% 992.2
1500-1999 8.2% 91.2% 993.4
2000-2499 8.6% 91.1% 997.0
2500-2999 9.0% 91.2% 1001.9
3000-3499 9.3% 91.2% 1004.4
3500-4000 9.8% 90.8% 1006.1
4000+ 10.4% 90.8% 1012.4

PDO varies from 983.5 up to 1012.4 depending on the group’s ice time.  This is largely driven by shooting percentage which varies from 7.5% to 10.4% with the players with the lowest amount of ice time having the lowest on-ice shooting percentage and the players with the most ice time having the highest shooting percentage.  Order is the enemy of luck so seeing shooting percentages ordered this nicely tells me something other than luck is happening.  Driving on-ice shooting percentage is a skill.  This means more talented players can have a natural PDO (the PDO that they should regress to) above 1000 and less talented players can have a nautral PDO below 1000.  Factor in the goaltending and a player could have a natural PDO well above or well below 1000.

Now, this is not to say that luck isn’t a factor in a players PDO, especially over small sample sizes, it’s just we can’t estimate that luck by assuming every players natural “regress to” PDO is 1000.  Daniel Sedin has a PDO of 1043 this season (through Thursday February 2nd).  Is it fair to suggest he has been luck and should see his PDO regress to 1000?  When you consider his4-year PDO is 1035 (and his 3 year PDO is 1054) probably not.  His natural, “regress to” PDO is probably not that far off his current 1043 PDO.  Now if you are talking about Todd Bertuzzi this season it’s a different story.  Through Thursday he had a a PDO of 1056 while his 4-year PDO is 994 and he hasn’t had a PDO above 1000 in any of the previous 3 seasons.  It is probably fair to presume that Bertuzzi’s natural regress to PDO is much closer to 1000, maybe even below 1000 in which case it is fair to conclude that Bertuzzi has probably been quite lucky so far this season and is unlikely to continue at this pace the remainder of the season.

When used properly PDO can be an indication of luck but to do so we need to consider the context of a players PDO, not just assume all players PDO’s will necessarily regress to 1000.

 

Jan 282012
 

I have been having a discussion as to whether shot quality exists over at Pension Plan Puppets and more precisely whether certain players can drive a teams shooting percentage while they are on the ice.  As part of the discussion I brought up the on-ice shooting percentage differences between Scott Gomez and Michael Cammalleri and decided that it would be useful to present that comparison as a post here.

First off, let me define shot quality as how I see it.  Shot quality is an ability for players to systematically drive (or suppress) shooting percentages when they are on the ice.  To me it doesn’t matter whether they can drive shooting percentages because they can get more shots from better shooting locations, or are better shooters, or are better playmakers setting up  changes with the goalie out of position.  Those are interesting things to investigate, but investigating them isn’t necessary to show shot quality exists.  Shot quality, in my mind, is all about a players being able to drive (or suppress) shooting percentage when they are on the ice, regardless of how.

In the past I have used examples such as Henrik Sedin vs Travis Moen and some comments I got were “but those are extreme cases” which is an interesting comment because in essence they person making that argument is admitting that shot quality exists but only in extreme cases.  So, I decided that it might be useful to take a look at two players who generally speaking play similar roles.  Scott Gomez and Michael Cammalleri.  Both Gomez and Cammalleri are top six forwards generally thought of as more offensive players.  What is also interesting is they over the past 4 1/2 seasons they both have switched teams and they have both spent a couple years playing on the same team, sometimes on the same line.    Let’s take a look at their 5v5 on-ice shooting percentages over the past 4 1/2 seasons.

 Sh% Gomez Cammalleri Difference
2007-08 7.09 8.15 1.06
2008-09 6.15 9.25 3.10
2009-10 7.89 9.66 1.77
2010-11 4.50 7.07 2.57
2011-12 7.96 8.11 0.15

In each and every season Cammalleri has had a higher shooting percentage, sometimes much higher.  Only this season have they been close in their on-ice shooting percentages.  If that isn’t a systematic ability by Cammalleri and his linemates to get a higher shooting percentage than Gomez and his linemates, I don’t know what is.  They can do it every singles season.

Now, let’s take a look at their offensive fenwick rates.  Here are their fenwick for per 20 minutes of 5v5 ice time rates.

 FF20 Gomez Cammalleri Difference
2007-08 15.86 14.3 -1.56
2008-09 16.76 15.38 -1.38
2009-10 14.21 13.4 -0.81
2010-11 16.4 14 -2.4
2011-12 16.8 12.06 -4.74

Well now, that tells us a different story.  Gomez and his line mates take far more shots than Cammalleri and his line mates, and they do it every single season.  Gomez and his line mates seem to have a much better skill at taking shots, but Cammalleri and his line mates seem to have a much better skill at capitalizing on shots.  The question now is, which skill results in more goals.  Here are their 5v5 goals for per 20 minute stats.

 GF20 Gomez Cammalleri Difference
2007-08 0.792 0.801 0.009
2008-09 0.757 1.020 0.263
2009-10 0.837 0.927 0.090
2010-11 0.534 0.713 0.179
2011-12 0.854 0.756 -0.098

Now that is interesting.  Cammalleri and his line mates have out produced Gomez and his line mates every year until this season.  Based on this one example, being able to drive shooting percentage resulted in more goals being scored than being able to drive shots.  If you were down by a goal in the third period, who would you rather have on the ice, Gomez and his line mates or Cammalleri and his line mates?

And the above is a perfect example of why I don’t like pure corsi/fenwick based evaluation of players.  If you just look at corsi/fenwick, Gomez looks like a very good player (see here and here), and Cammalleri does not.  But, if you look at goals, over the past 2 seasons 54.1% of all goals scored while Cammalleri was on the ice were for the Canadiens while just 47.2% of all goals scored while Gomez was on the ice were for the Canadiens.  Who is the better player, and who would I rather have on my team?  Cammalleri by a country mile.

Let’s take it one step further and how they played when they were on the ice together and when they were apart over the past 2 seasons.

Together Cammalleri Gomez
GF% 54.8% 53.9% 45.4%
Corsi% 52.3% 47.9% 51.6%

Wow, that is dramatic.  When they play together can an drive shots (corsi) and goals.  When Cammalleri is not playing without Gomez he can drive goals, but not shots (corsi) and when Gomez is playing without Cammalleri he can drive shots (corsi) but not goals.  Again, who would you rather have on your team?  For me, I’ll take the guy who can drive goals thank you very much.

And that my friends, is a perfect example of when a corsi based analysis will fail.

 

Dec 092011
 

Gabe Desjardins of Arctic Ice Hockey asks the question about whether a player can influence his teammates shooting percentage.  To answer this question he took a look at the Pittsburgh Penguins shooting percentages with and without Mario Lemieux.  The conclusion:

I’d posit that Lemieux’s playmaking contribution is about as large as we’re going to consistently find – something on the order of 7-8% – and we can use it to bound the impact that a player can truly have on the quality of his teammates’ scoring chances.

Since I have the numbers handy I figured I’d take a look at some more recent examples but instead of looking at straight shooting percentage I looked at corsi shooting percentage (since I had corsi data more available).  Corsi shooting percentage is simply goals for divided by corsi for.  I’d consider Joe Thornton one of the premiere playmaking centers in the league today so let’s take a look at how some players performed while playing with, and without, him.

CSH% With Thornton CSH% Without Boost
Marleau 4.93% 3.88% 27.02%
Setogutchi 5.09% 3.80% 33.82%
Heatley 5.59% 4.32% 29.50%

I included the past 4 years with Marleau, 3 years with Setogutchi and 2 years with Heatley.  That would indicate Thornton has an approximately 30% boost in corsi shooting percentage to his teammates.  Certainly far more than the 8% Gabe predicted as the upper bound.

Now, let’s take a look at another great player, Sidney Crosby.

CSH% With CSH% Without Boost
Malkin 7.30% 5.15% 41.89%
Dupuis 5.73% 4.06% 40.95%
Fleury 5.75% 4.22% 36.15%

All players are using 4 years of data.  I included Fleury in the list because it provides a good proxy of the Penguins shooting percentages when Crosby is on the ice vs when he is not.  This would seem to indicate that Crosby is worth a nearly 40% boost in his teams shooting percentage.  That’s significantly more than even Thornton and a massive amount more than Gabe’s estimated upper bound.  Maybe we should revise the upper bound to be 40%, not 8%.

For interest sake, here how much Crosby influenced his teammates corsi rates.

Boost in CF20
Malkin 21.15%
Dupuis 18.15%
Fleury 15.59%

While a ~20% boost is significant, it is at best only half the boost he provided to corsi shooting percentage.  Driving shooting percentage is a more significant reason why Crosby is so good offensively than driving corsi events.

 

Update:  Eric over at Broad St. Hockey has an interesting post looking at individual shooting percentages as opposed to on-ice shooting percentages as I did above.  Four of the players he looked at are H. Sedin, Crosby, Thornton and Datsyuk and for each he looked at a number of teammates with at least 30 shots with and without.  Taking it a step further I think it is necessary to average across players to get a better idea of what is happening.  If you do that, this is what you get:

With Without Boost
Sedin 11.19% 6.81% 64.28%
Crosby 9.14% 7.55% 21.17%
Thornton 9.69% 7.27% 33.19%
Datsyuk 9.36% 6.98% 34.09%

Wow, that might make Sedin the best playmaker in the league, by a significant margin.  Crosby doesn’t look quite as good as my “on-ice” analysis but that is because much of the reason why Crosby improves his linemates on-ice shooting percentage is because he is such a great shooter himself.

The point still stands, without considering shooting percentages we aren’t getting anywhere close to having a complete analysis of a players impact on the game.

 

Nov 222011
 

I hate to keep beating the “Shooting Percentage Matters” drum but it really dumbfounds me why so many people choose to ignore it, or believe it is only a small part of the game and not worth considering and instead focus their attention on corsi/fenwick, and corsi/fenwick derived stats as their primary evaluation too.

It dumbfounds me that people don’t think players have an ability to control shooting percentage yet we all seem to agree that shooting percentage is affected by game score.  Rob Vollman wrote the following in a comment thread at arctic ice hockey.

<blockqote>The score can affect the stats because teams behave differently when chasing or protecting a lead…</blockquote)

He isn’t specifically referring to shooting percentage, but shooting percentage varies based on game score and I think most people accept that.  So, while people freely accept that teams can play differently depending on score, they seemingly choose not to believe that players can play different depending on their role, or skillset.  Or rather, it isn’t that they don’t believe players can play differently (for example they realize there are defensive specialists) they just choose not to accept that a players style of play (in addition to their talents, which often dictates their style of play) will affect their stats, including shooting percentage.  An example, which I brought up at The Puck Stops Here is Marian Gaborik vs Chris Drury.  Both Gaborik and Drury played the past 2 seasons on the NY Rangers but Gaborik played an offensive role and Drury generally played a more defensive/3rd line role.  As a result, here are their offensive stats at 5v5 over the past 2 seasons.

Gaborik Drury Gaborik’s Edge
Team Fenwick For per 20min WOI 13.8 12.8 +8%
Team Sh% For WOI 10.26% 6.18% +66%
Team Goals For per 20 min WOI 1.031 .575 +79%

Shooting percentage took what was a slight edge for Gaborik in terms of offensive fenwick for and turned it into a huge advantage in goals for.  Part of that is Gaborik and his line mates better skill level and part of it is their aggressive offensive style of play, but regardless of why, we need to take shooting percentage into account or else we will undervalue Gaborik at the offensive end of the rink and over value Drury.

It isn’t just Gaborik and Drury whose offense is significantly impacted by shooting percentage.  It happens all the time.  I took a look at all players that had 2000 5v5 even strength on-ice offensive fenwick events over the past 4 seasons.  From there I calculated their expected on-ice goals scored based on their ice time using league-wide average  on-ice fenwick for per 20 minutes (FF20) and league-wide average fenwick shooting percentage (FSH%).

I next calculated an expected goals based on the league-wide FF20 and the players FSH% as well as an expected goals based on the players FF20 and the league-wide average FSH%.  When we compare these expected goals to the expected goals based solely on the league-wide average we can get an idea of whether a players on-ice goal production is driven mostly by FF20 or FSH% or some combination of the two.

The following players had their on-ice 5v5 goal production influenced the most positively or most negatively due to their on-ice 5v5 FSH%.

Player Name %Increase from FSH%
MARIAN GABORIK 40.6%
SIDNEY CROSBY 36.3%
ALEX TANGUAY 33.1%
HENRIK SEDIN 32.8%
BOBBY RYAN 32.5%
EVGENI MALKIN 31.9%
DANIEL SEDIN 31.6%
ILYA KOVALCHUK 30.6%
NATHAN HORTON 29.6%
J.P. DUMONT 29.4%
GREGORY CAMPBELL -12.4%
RYAN CALLAHAN -13.9%
RADEK DVORAK -15.6%
CHRIS DRURY -16.8%
SEAN BERGENHEIM -19.4%
SCOTT GOMEZ -19.7%
MARTIN HANZAL -21.5%
MIKE GRIER -21.5%
DANIEL WINNIK -24.5%
TRAVIS MOEN -32.1%

And the following players had their on-ice 5v5 goal production influenced the most positively or most negatively due to their on-ice 5v5 FF20.

Player Name %Increase from FF20
HENRIK ZETTERBERG 24.7%
ALEX OVECHKIN 21.7%
PAVEL DATSYUK 20.6%
TOMAS HOLMSTROM 19.9%
NICKLAS BACKSTROM 19.8%
ERIC STAAL 19.7%
RYANE CLOWE 18.8%
ALEXANDER SEMIN 18.3%
SCOTT GOMEZ 18.0%
ZACH PARISE 17.9%
MARTY REASONER -6.5%
ANDREW COGLIANO -6.5%
ANTTI MIETTINEN -6.7%
KYLE BRODZIAK -7.3%
CHRIS KELLY -8.6%
ILYA KOVALCHUK -9.8%
JAY MCCLEMENT -10.4%
MICHAL HANDZUS -14.4%
JOHN MADDEN -14.5%
TRAVIS MOEN -15.6%

Some interesting notes:

  1.  The range in the influence of FSH% is significantly larger than the range of influence of FF20 indicating that shooting percentage is more important than shot generation in terms of scoring goals.
  2. The FSH% list is not random.  The list is stratified.  Offensive players at the top, non-offensive players at the bottom (plus Scott Gomez who gets offensive minutes, but sucks).  What you see above is not luck.  There is order to the list, not randomness.
  3. Speaking of Gomez, he sucks at on-ice FSH%, but has a very good FF20, though that is partly due to offensive zone start bias.
  4. Ilya Kovalchuk is the anti-Gomez.  He has a great FSH%, but is horrible at helping his team generate shots.
  5. The standard deviation of the FSH% influence is 14.5% while it is 8.3% for FF20 influence so it seems FSH% has a much greater influence on scoring goals than FF20.  This is not inconsistent with some of my observations in the past or observations of others.

So, what does all this mean?  Shooting percentage matters, and matters a lot and thus drawing conclusions based solely on a corsi analysis is flawed.  It isn’t that generating shots and opportunities isn’t important, but that being great at it doesn’t mean you are a great player (Gomez) and being bad at it doesn’t make you a bad player (Kovalchuk).  For this reason I really cringe when I see people making conclusions about players based on a corsi analysis.  A corsi analysis will only tell you how good he is at one aspect of the game, but is not very good at telling you the players overall value to his team.  My goal is, and always will be, to try and evaluate a players overall value and this is why I really dislike corsi analysis.  It completely ignores a significant, maybe the most significant, aspect of the game.  Furthermore, I believe that offensive ability and defensive ability should be evaluated separately, which many who do corsi analysis don’t do or only partially or subjectively do.

I really don’t know how many different ways I can show that shooting percentage matters a lot but there are still a lot of people who believe players can’t drive or suppress shooting percentage or believe that shooting percentage is a small part of the game that is dwarfed by the randomness/luck associated with it (which is only true if sample size is not sufficiently large).  The fact is corsi analysis alone will never give you a reliable (enough to make multi-million contract offers) evaluation of a players overall ability and effectiveness.  Shooting percentage matters, and matters a lot.  Ignore at your peril.

 

Oct 272011
 

There has been a fair bit of discussion going on regarding shot quality the past few weeks among the hockey stats nuts.  It started with this article about defense independent goalie rating (DIGR) in the wall street journal and several others have chimed in on the discussion so it is my turn.

Gabe Desjardins has a post today talking about his hatred of shot quality and how it really isn’t a significant factor and is dominated by luck and randomness.  Now, generally speaking when others use the shot quality they are mostly talking about thinks like shot distance/location, shot type, whether it was on a rebound, etc.  because that is all data that is relatively easily available or easily calculated.  When I talk shot quality I mean the overall difficulty of the shot including factors that aren’t measurable such as the circumstances (i.e. 2 on 1, one timer on a cross ice pass, goalie getting screened, etc.).  Unfortunately my definition means that shot quality isn’t easily calculated but more on that later.

In Gabe’s hatred post he dismisses pretty much everything related to shot quality in one get to the point paragraph.

 

Alan’s initial observation – the likelihood of a shot going in vs a shooter’s distance from the net – is a good one.  As are adjustments for shot type and rebounds.  But it turned out there wasn’t much else there.  Why?  The indispensable JLikens explained why – he put an upper bound on what we could hope to learn from “shot quality” and showed that save percentage was dominated by luck.  The similarly indispensable Vic Ferrari coined the stat “PDO” – simply the sum of shooting percentage and save percentage – and showed that it was almost entirely luck.  Vic also showed that individual shooting percentage also regressed very heavily toward a player’s career averages.  An exhaustive search of players whose shooting percentage vastly exceeded their expected shooting percentage given where they shot from turned up one winner: Ilya Kovalchuk…Who proceeded to shoot horribly for the worst-shooting team in recent memory last season.

So, what Gabe is suggesting is that players have little or no ability to generate goals aside from their ability to generate shots.  Those who follow me know that I disagree.  The problem with a lot of shot quality and shooting percentage studies is that sample sizes aren’t sufficient to draw conclusions at a high confidence level.  Ilya Kovalchuk may be the only one that we can say is a better shooter than the average NHLer with a high degree of confidence, but it doesn’t mean he is the only one who is an above average shooter.  It’s just that we can’t say that about the others at a statistically significant degree of confidence.

Part of the problem is that goals are very rare events.  A 30 goal scorer is a pretty good player but 30 events is an extremely small sample size to draw any conclusions over.  Making matters worse, of the hundreds of players in the NHL only a small portion of them reach the 30 goal plateau.  The majority would be in the 10-30 goal range and I don’t care how you do your study, you won’t be able to say much of anything at a high confidence level about a 15 goal scorer.

The thing is though, just because you cannot say something at a high confidence level doesn’t mean it doesn’t exist.  What we need to do is find ways of increasing the sample size to increase our confidence levels.  One way I have done that is to use 4 years of day and instead of using individual shooting percentage I use on-ice shooting percentage (this is useful in identifying players who might be good passers and have the ability to improve their linemates shooting percentage).  Just take the list of forwards sorted by on-ice 5v5 shooting percentage over the past 4 seasons.  The top of that list is dominated by players we know to be good offensive players and the bottom of the list is dominated by third line defensive role players.  If shooting percentage were indeed random we would expect some Moen and Pahlsson types to be intermingled with the Sedin’s and Crosby’s, but generally speaking they are not.

A year ago Tom Awad did a series of posts at Hockey Prospectus on “What Makes Good Players Good.”  In the first post of that series he grouped forwards according to their even strength ice time.  Coaches are going to play the good players more than the not so good players so this seems like a pretty legitimate way of stratifying the players.  Tom came up with four tiers with the first tier of players being identified as the good players.  The first tier of players contained 83 players.  It will be much easier to draw conclusions at a high confidence level about a group of 83 players than we can about single players.  Tom’s conclusions are the following:

The unmistakable conclusions from this table? Outshooting, out-qualitying and out-finishing all contribute to why Good Players dominate their opponents. Shot Quality only represents a small fraction of this advantage; outshooting and outfinishing are the largest contributors to good players’ +/-. This means that judging players uniquely by Corsi or Delta will be flawed: some good players are good puck controllers but poor finishers (Ryan Clowe, Scott Gomez), while others are good finishers but poor puck controllers (Ilya Kovalchuk, Nathan Horton). Needless to say, some will excel at both (Alexander Ovechkin, Daniel Sedin, Corey Perry). This is not to bash Corsi and Delta: puck possession remains a fundamental skill for winning hockey games. It’s just not the only skill.

In that paragraph “shot quality” and “out-qualitying” is used to reference a shot quality model that incorporates things like shot location, out-finishing is essentially shooting percentage, and outshooting is self-explanatory.  Tom’s conclusion is that the ability to generate shots from more difficult locations is a minor factor in being a better player but both being able to take more shots and being able to capitalize on those shots is of far greater importance.

In the final table in his post he identifies the variation in +/- due to the three factors.  This is a very telling table because it tells it gives us an indication of how much each factors into scoring goals.  The following is the difference in +/- between the top tier of players and the bottom tier of players:

  • +/- due to Finishing:  0.42
  • +/- due to shot quality:  0.08
  • +/- due to out shooting:  0.30

In percentages, finishing ability accounted for 52.5% of the difference, out shooting 37.5% of the difference and shot quality 10% of the difference.  Just because we can’t identify individual player shooting ability at a high confidence level doesn’t mean it doesn’t exist.

If we use the above as a guide, it is fair to suggest that scoring goals is ~40% shot generation and ~60% the ability to capitalize on those shots (either through shot location or better shooting percentages from those locations).  Shooting percentage matters and matters a lot.  It’s just a talent that is difficult to identify.

A while back I showed that goal rates are better than corsi rates in evaluating players.  In that study I showed that with just 1 season of data goal for rates will predict future goal for rates just as good as fenwick for rates can predict future goal for rates and with 2 years of data goal for rates significantly surpass fenwick for rates in terms of predictability.  I also showed that defensively, fenwick against rates are very poor predictors of future goal against rates (to the point of uselessness) while goals against rates were far better predictors of future goal against rates, even at the single season level.

The Conclusion:  There simply is no reliable way of evaluating a player statistically at even a marginally high confidence level using just a single year of data.  Our choices are either performing a Corsi analysis and doing a good job at predicting 40% of the game or performing a goal based analysis and doing a poor job at predicting 100% of the game.  Either way we end up with a fairly unreliable player evaluation.  Using more data won’t improve a corsi based analysis because sample sizes aren’t the problem, but using more data can significantly improve a goal based analysis.  This is why I cringe when I see people performing a corsi based evaluation of players.  It’s just not, and never will be, a good way of evaluating players.

 

Oct 102011
 

There has been an interesting discussion of on-ice shooting percentage at Tyler Dellow’s mc79hockey.com.  I have argued that we need to look at on-ice shooting percentage as a talent, and not something that just happens randomly while others have largely dismissed it.  One person in particular is Gabe Desjardin’s who has a followup post on his blog largely dismissing its importance.

In his blog post Gabe first discusses Gaborik’s value just considering his on-ice shooting percentage.

So are these totals 75% skill then?  Let’s do a quick check on how many goals that skill would be worth: 1000 on-ice shots/season * 2.5% above mean * 75% = 18.75 goals above average.  Double that to get to an approximate replacement level of 37.5 goals or just over six wins.  The current price for one win on the free agent market is roughly $3M, so we’d estimate Gaborik’s offensive value at more than $18M.

Gabe doesn’t believe any one player could be worth $18M based just on shooting percentage so he tries to shoot a hole in that by looking at 2yr vs 2yr regression.

Needless to say, anytime you come up with a metric that says a player should get paid $18M, you have to go back and check your math.  I did that by splitting the last four years into two two year periods (2007-08/2008-09 vs 2009-10/2010-11) and comparing on-ice shooting percentage among players who had 1000+ on-ice shots in each period.  I found that player on-ice shooting regressed 80% to the mean from the first set to the second, which puts Gaborik’s apparent talent closer to $5M.

Ok, so we have Gaborik’s value down to $5M.  That still seems pretty large to me but Gabe dismisses that further by suggesting some of that $5M has to be attributed to his linemates, arena bias, and strangely, a players opponents (particularly at home).

This is a key point, of course, and one that may not come through when we talk about team-level effects or try to figure out the value of individual top six forwards: when a #1 line plays against a #4 line, their shooting percentage goes up relative to when they’re playing power-vs-power.

Here is the thing.  The whole reason I participate in these debates is to suggest that players do in fact have the ability to drive or suppress shooting percentage and thus we must consider shooting percentage, in addition to corsi, when evaluating players.  So, it amazes me when I am debating someone that is trying to minimize the ability to drive/suppress shooting percentage that they bring up such observations that a players shooting percentage will go up when they are playing weaker players (i.e. the fourth line) who I presume can’t suppress shooting percentage as well as the stronger players.   There is clearly something wrong with the logic there.  Players don’t have the ability to suppress shooting percentage, but fourth liners are worse at suppressing shooting percentage than first liners??

The other thing I want to discuss is Gabe’s calculation that shooting percentage regresses 80% to the mean based on his 2yr vs 2yr calculation.  I won’t dispute his math because it is probably true, but I will suggest that I don’t believe that a league-wide observation can be applied to individual players.  There are a number of factors that influence a players on-ice shooting percentage.

1.  The quality of his linemates.

2.  The quality of opposition.

3.  Style of play (i.e.  aggressive offensive game vs defensive style).

4.  Score effects

Team building generally revolves around a small number of players.  Pittsburgh has Crosby, Malkin and Staal as their core forwards, everyone else is pretty much interchangeable.  Pretty much every team is like this.  A lot of those interchangeable parts move from team to team or even line to line on the same team or get asked to play different roles on the same team.  An injury to Kunitz and Pascal Dupuis gets bumped from the third line to playing with Crosby.  For these mostly interchangeable parts there can be a lot of variation in who they play with, the team they play on, and who they play against, and the roles they are asked to play.  All these factors are at play when Gabe calculates his 80% regression to the mean.  The good players who have well-defined roles don’t see near the same variation in their on-ice shooting percentages.  Look at the Crosby’s and Gaborik’s.  They are consistently at the top of the list.  Look at the Moen’s and Marchant’s and Pahlsson’s, they are consistently near the bottom of the list.  The players we perceive as good offensive players are at the top of the list.  The players we perceive as weak offensive players, or defensive minded players, are at the bottom of the list.  That’s a talent that we must consider.

 

 

Jul 192011
 

An interesting statistical debate sprung up today started by Tom Benjamin who wrote about his skepticism of the Corsi statistic.  In it Tom comments on the fact that Ryan Kesler and Ryan Clowe ranked so highly in corsi in response to Greg Ballentine’s posts at The Puck Stops Here.

Greg’s examples, it seems to me, make a good case against the Corsi statistic. First, both the Kesler and Clowe stories tell us how much influence context has – neither Kesler nor Clowe could have done it playing on a different team or even playing in a different role on the same team. In other words, this is not really an individual statistic.

Of course, this got some in the Corsi crowd up in arms and states that Corsi can’t be used on its own without considering its context.  Gabe Desjardin’s comments on Tom’s post with the following:

Corsi, like any other statistic, needs to be understood in the context of other factors. Sneering at it because, like any other simple statistic, it doesn’t provide a unified measure of a player’s complete value doesn’t contribute anything to the larger discussion of hockey analysis.

Ok, so I am glad we have that cleared up.  Corsi is just a stat without meaning unless you consider the context.  Oh good, now it is on par with nearly every other stat in hockey.  Unfortunately people actually use corsi to actually draw conclusions about players.

Here is the thing that really irks me about some in the Corsi crowd.  They just assume that shot quality doesn’t exist.  An anonymous commenter using the name ‘Name’ writes the following:

Even the basis of corsi, that shot quality always evens out, so we just have to measure shot quantity, is inherently flawed. The strategies and styles some teams play lead to giving up a greater number of shots, but reducing quality ones, whereas some teams strive to block or prevent every single shot, no matter where it comes from. Therefore some players, just by playing on a certain time, will inherently be on the ice for more shots against. It doesn’t mean they’re giving up more quality scoring chances, or making lots of defensive mistakes, or failing to control play offensively.

To this comment Gabe comes back with his favourite response to any challenge put his way:

Name these teams and players. Thanks.

Now that is a fair response, unfortunately he ignores anyone who actually names these players.  The reality is shot quality doesn’t even out.  Some players drive shot quality and some players suppress it.  Some players have a significantly different +/- than corsi +/- year in and year out.  I gave an example the other day in Brendan Morrison.  Here are some other names for Gabe to consider.

Some guys who can drive shooting percentage: Sidney Crosby, Marian Gaborik, Nathan Horton, Bobby Ryan, Martin St. Louis.   Henrik and Daniel Sedin.  Alex Tanguay.  Jason Spezza.

Some guys who can suppress shooting percentage:  Marco Sturm, Travis Moen, Tyler Kennedy, Taylor Pyatt, Shawn Thornton, Chris Drury, Jeff Carter. Torrey Mitchell. Kamil Kreps.

No one in the second group had an opposition shooting percentage above 7.6% in any single season over the past 4 years.  Only a handful of times over the past 4 years has any of the players in the first list had an on ice shooting percentage below 9% and only Bobby Ryan’s 23 game 2007-08 season was below 7.6%.

Now, you’ll probably notice that the first group are all first line players who are expected to produce offense while the second group are mostly third line players asked to shut down the opposition.  It’s difficult to suggest it was just luck that this is how things panned out.  No, some combination of talent and style of play will affect your on ice shooting and opposition shooting percentages.  And again it needs to be stated that shooting percentage is much more highly correlated with scoring goals than corsi.

Jeff Carter is an especially interesting case in that you can in no way argue that he has played in front of especially good goaltending that would drive down his shooting percentage against and yet he has a really slow shooting percentage and yet has one of the highest corsi against of any forward over the past 4 seasons (20.1 corsi events against per 20 minutes over the past 4 seasons ranks 300 of 310) but his goals against (0.753 per 20 minutes)  is decidedly average or even slightly better than average (ranks 139 of 310).

So Gabe, those are some players for you to consider.  I look forward to your response.

Mar 182011
 

The guys over at Behind the Net have initiated a ‘prove shot quality exists’ competition and in response to that Rob Vollman took a quick and dirty look at shooting percentage suppression.  As I showed the other day, Rob’s logic was a little off.

Rob started off by identifying a number of players with high on ice save percentages over the past 3 seasons.  Some of these guys included low minute players mostly playing on the fourth line against other fourth line caliber players, but there were a handful of players who played relative significant number of minutes and still put up good on ice save percentages.  Let me remind you of a few names that Rob identified:  forwards Marco Sturm, Manny Malhotra, Tyler Kennedy, Travis Moen, Taylor Pyatt, Michael Ryder, defensemen Kent Huskins, Sean O’Donnell, Mike Weaver, Mark Stuart.  I’ll get back to these guys later but I’ll claim that Rob dismissed some of them prematurely by claiming they played against weak competition.

As you may or may not know I have developed offensive and defensive ratings for every player and these can be found at http://stats.hockeyanalysis.com/ Furthermore, I have created these using goals for/against as well as shots for/against, fenwick for/against, and corsi for/against.  For clarification, fenwick is shots + missed shots while Corsi is shots + missed shots + blocked shots.  For this study I decided to use fenwick instead of shots because I had the data handy and I was too lazy to get the shot data in the right format but there shouldn’t be a significant difference (the two are very highly correlated).

Continue reading »

Mar 152011
 

I thought this debate had been fully hashed out already but apparently some people still don’t believe that the game score has an impact on shooting percentage (and shot quality).  The following table shows the shooting percentages by game score over the past 3 seasons (2007-08 to 2009-10) during even strength situations where neither goalie is pulled for any reason (including delayed penalty situations).

Situation Shots Goals SH% Prob<= Prob>
Down2+ 23650 1852 7.83 0.3794 0.6206
Down1 30447 2356 7.74 0.1696 0.8304
Tied 60753 4427 7.29 0.0000 1.0000
Up1 26842 2288 8.52 0.9999 0.0001
Up2+ 19351 1779 9.19 1.0000 0.0000
Overall 161043 12702 7.89 0.5024 0.4976

The Situation, Shots, Goals, and SH% columns are self explanatory.  As you can see, shooting percentage is at its lowest in game tied situations, increases slightly for teams that are trailing and increases significantly for teams that are leading.

The second last column titled Prob<= show the probability (according to a binomial distribution) that that number of goals or fewer would be scored on that number of shots if the expected shooting percentage was 7.89%, the same as the overall 5v5 shooting percentage.  The last column titled Prob> is simply 1-Prob<= and shows the probability of getting more than that number of goals on that number of shots.  So, in down 2+ goal situations, there is a 37.94% chance of their being 1852 or fewer goals scored on 23650 shots which indicates that the down2+ shooting percentage isn’t different from the 5v5 mean at any reasonable confidence level.  The same conclusion can be drawn about down1 situations.  But, the shooting percentages in game tied, up1 and up2+ situations are statistically different at an extremely high confidence level.  Essentially there is zero chance that game tied, up1, or up2+ situations have the same natural shooting percentages as game overall 5v5 situations.  In no way can luck be the sole reason for these differences.

So, does this conclusively tell us that shot quality exists and varies according to game score?  It probably does, but I can’t say it is conclusive as it could mean that teams that trail a lot have bad goaltending (the reason they are trailing) and this results in the team leading having an inflated shooting percentage.  So, what if we looked at shots against a particular team.  Let’s say, for example, against the NY Rangers.  Here is what that looks like.

Situation Shots Goals SH% Prob<= Prob>
Overall 5159 386 7.48 0.5135 0.4865
Up1 843 73 8.66 0.9116 0.0884
Up2+ 485 46 9.48 0.9571 0.0429
Leading 1328 119 8.96 0.9800 0.0200
Tied 2004 138 6.89 0.1658 0.8342

I chose the Rangers because they use predominantly one goalie and that goalie is generally speaking a quality goalie.  As you can see, the confidence levels aren’t quite as strong as league wide mostly because of the smaller sample size but if we combine the up1 and up2+ categories we can say that shot quality against the Rangers when the opposing team is leading is statistically different than shooting percentage against the Rangers overall.

If you are interested in seeing what happens with a team that has had chronically bad goaltending, here is the same table for the Maple Leafs.  We see the same sort of things.

Situation Shots Goals SH% Prob<= Prob>
Overall 5309 491 9.25 0.5120 0.4880
Up1 938 94 10.02 0.8098 0.1902
Up2+ 906 100 11.04 0.9698 0.0302
Leading 1844 194 10.52 0.9712 0.0288
Tied 1985 149 7.51 0.0034 0.9966

So what have we learned.

  1. Shooting percentages vary according to game score.
  2. Those shooting percentage differences can’t be attributed to luck.
  3. Those shooting percentage differences can’t be attributed to goaltending.

That means, it must be the quality of the shots that varies across game scores.  In short, we can conclude that when teams get down in a game they open up and take more chances offensively which in turn gives up higher quality shots against which makes perfect sense to me.

When we combine this with my previous post on the Washington Capitals shooting percentage last season, it is probably safe to assume that shot quality exists and we can’t safely assume that all shots can be treated equal in all situations.

Jan 302011
 

Yesterday there was a post on the Behind the Net Blog which discussed the Washington Capital’s 2009-10 even strength shooting percentage of 11.0% and the conclusion was that it must be mostly luck which resulted in a shooting percentage that high.  But was it?  It was noted in the article that in 2007-08 the Capitals shot at 8.1%, in 2008-09 they shot at 8.2% and this season they are shooting at 8.2% again.  So clearly 2009-10 appears to be an anomaly, but was it a luck driven anomaly or something else?

Most people in the hockey analysis world have been using a simple binomial distribution to simulate luck so I’ll do that here too.  The thing is, if the Washington Capitals were really a 8.2% shooting team last year, the chances of them shooting 11.0% or better on 2045 shots is a mere 0.0042%.  That kind of luck we should expect once every 8000 NHL seasons.  In short, we can be pretty confident that the Capitals 11.0% shooting percentage wasn’t all luck driven.

So the next question is, how much of it is luck, and how much can we attribute to other factors?  Well, let’s assume that their good luck was significant to the point where there would only be a 5% chance they could have experienced even more luck.  We can do this by constructing a binomial distribution using centered on a shooting percentage where the chance of producing a shooting percentage of >11.0% is 5%.  The result is shown in the following chart:

The far left vertical line is the number of goals that Washington would produce if they had an 8.2% shooting percentage and the far right line is their actual shooting percentage.  The center vertical line is the theoretical shooting percentage we would need to meet the 5% luck conditions outlined above.  Under this scenario one could suggest of the extra 57 goals that Washington scored above what they would get if they shot at 8.2%, 22 of those goals can be attributed to luck and 35 can be attributed to skill.

But what if we assumed the Capitals were extremely lucky and there was only 1% chance of having greater luck.  Under that scenario their true talent level would be 9.49% shooting percentage and 26 goals would be due to skill and 31 would be due to luck.

Regardless of how you want to look at it, a significant portion of the Capitals elevated shooting percentage was likely due to non-luck factors, be they actual talent, playing style, score effects, etc.