Yesterday I wrote about Brandon Sutter and his seemingly very good defensive ability in response to @DTMAboutHeart‘s comment that he was a below replacement level defensive player. The counter argument that @DTMAboutHeart (and others) put put is that Save % is all random and it infects goals against data with mass randomness that will inherently lead to outcomes like Sutter’s goals against stats that have no basis in talent. Exhibit A of this yesterday came in the form of a quote from Warren Buffett.
Here is Warren Buffett’s take on the idea NHL players can influence SV% pic.twitter.com/LGwlozW3wD
— DTM About Heart (@DTMAboutHeart) July 15, 2016
This is a wonderful quote and gets to the idea of randomness quite well. Yes, randomness and luck can lead to success that is not based on any specific individual skill or talent and we must be aware of that.
Having not read the context of that quote I can only assume that it was in reference to investing in stocks and/or businesses. I will assume he is trying to say that some investors will get lucky and make a ton of money in the stock market, possibly in reference to day traders or short sellers. I that his overall argument was not that investing in the stock market is all random and akin to playing the lottery but rather he went on to suggest that a a sound, diligent, long-term investment strategy was the best strategy for maximising your return on investment. This is what Warren Buffett is known for and it has brought him great wealth.
How does this relate to hockey? Well, sure, there is luck and randomness involved in shooting and save percentages and that can lead to successes (or failures) that are not based on talent (or lack of it). However that does not mean all successes (or failures) are due to luck and randomness. Happily we have tests for this.
The following table shows a comparison of Brandon Sutter’s on-ice save percentage for 6 seasons along side his teams save percentage for the same 6 seasons (these are the 6 seasons in which Sutter had >500min of ice time). Also included are the goals and shots against when Sutter is on the ice as well as the likihood that Sutter’s on-ice save percentage is due to randomness given an expected save percentage of his teams save percentage. Also shown are his 6-year totals.
|Season||Team Sv%||Sutter Sv%||SA||GA||Likelihood Sutter’s on-ice Sv% is due to randomness alone.|
For any given season the likelihood of Sutter achieving his on-ice save percentage due solely to randomness ranges between 1.90% and 46.78%. If we were just looking at one or two seasons we would not be able to assume that this was not random. However when we look at six seasons in aggregate we see that there is only a 0.06% chance that Sutter’s on-ice save percentage is due solely to randomness.
To put this another way we would expect Brandon Sutter’s 6-year save percentage to occur once for every ~1800 players. However, during that 6-year span there were just 126 skaters and only 45 forwards who were on the ice for as many shots against (2815) as Brandon Sutter was. Brandon Sutter’s outstanding on-ice save percentage is almost certainly not driven by randomness. End of story.
With that said, I stand by my comment that calling Brandon Sutter a below replacement level defensive player should be used as a litmus test for whether you are doing hockey analytics right. It is akin to Sidney Crosby and evaluating players offensive ability. If your analytics doesn’t suggest that Sidney Crosby is at or near the top of the league in offensive ability you are probably doing it wrong.
Exceptions will exist to all trends. Hard part is separating which ones are due to randomness or true talent.
— Garret Hohl (@GarretHohl) July 15, 2016
Luck and talent are not mutually exclusive. They can, and normally do, coexist within the same system. As Garret Hohl says, isolating talent and luck from each other can be difficult however that is not an excuse for not attempting to try, or worse, just assuming the talent portion doesn’t exist or matter so it makes your life easier.