Dec 162010
 

In the Hockey Statistical Analysis world Tomas Vokoun is an interesting case study because depending on how he gets evaluated he either shows up as an very good goalie or in some cases a true elite goalie in the NHL.  Most ways we evaluate goalies has to do with save percentages.  We either look at overall save percentage or even strength save percentage or even even strength game tied save percentage.  Under all of these scenarios Vokoun excels to various degrees.  A recent Behind the Net Hockey Blog post asked several hockey statistic analysts to discuss “elite goalies” and Tomas Vokoun’s name came up frequently.  What is dumbfounding to me is Vokoun’s record because his won-loss record (79-80-25) is notably worse over the past 3 seasons than his backups (32-22-8).  That can’t be a sign of an elite goalie, even if his backups have been relatively good (i.e. Craig Anderson).  One may postulate it is due to facing tougher competition as backup goalies often get the to play against weaker teams or one may postulate it is just due to bad luck.  Or maybe, he just isn’t a great goalie.

Since shots totals and shooting/save percentage is often affected by game score I’ll focus on 5v5 even strength game tied statistics to balance everything out.  Over the last 3 seasons (2007-08 to 2009-10) there are 35 goalies with 1500 or more 5v5 game tied minutes.  Of these goalies, Tomas Vokoun ranks 8th in 5v5 game tied save percentage which may not be elite, but still very good.  Jonas Hiller tops the list with a .942 save % with Vokoun at .933 and Chris Osgood trails the list with a .906 save %.  So, Vokoun looks pretty good.

But, Tomas Vokoun ranks just 23rd in goals against average which isn’t great and probably average at best.  Those who are in love with fenwick numbers will note that Vokoun has the second highest fenwick against of any goalies with 1500+ 5v5 tied minutes and he gives up so many goals because Florida gives up so many shots and scoring chances.  Of course, I believe that not all shots against are equal and shot totals can be influenced by style of play as much as talent.  If you don’t believe style of play affects shot totals and scoring chances, ask yourself why there are score effects on shot/corsi totals?  The answer is depending on the score, teams play differently.  But teams play differently when the score is tied as well.  Some teams play a defense first style, even when game is tied, and others play a more wide open offensive style.  Florida, without any true elite offensive stars, probably plays more of a defensive game which would naturally lead to more shots against, but not necessarily more quality scoring chances against.

So yes, Florida gives up a lot of shots, but how good is Tomas Vokoun’s competition really.  He does play in the weakest division in the NHL and yet he can’t produce a good won-loss record.  Just looking at Vokoun’s opposition, his opponents rank dead last in goals for per 20 minutes so compared to other goalies he is playing against relatively weak opponents offensively.  His oppositions GF% (goals for / goals for + against) is also fourth worst so overall so he plays against very weak opposition in terms of scoring goals and stopping goals.  For those who prefer Fenwick, his opposition has a FF% (fenwick for / fenwick for + against) of .499, good for 27th among the 35 goalies.  So his opposition isn’t good and his performance in goals against average isn’t good either.  That isn’t a good combination if you want to be considered an elite level goalie.

How about a direct comparison with his backups.  In 2007-08 his goals against average per 20 minutes was significantly worse than Craig Anderson’s (0.949 for Vokoun, 0.538 for Anderson) while Anderson’s opponents had a slightly better goals for per 20 minutes (0.678 vs 0.671).  In 2008-09 Vokoun had a much better season giving up 0.697 goals per 20 minutes compared to Anderson’s 0.896 though Anderson played against slightly better offensive competition.  In 2009-10 Vokoun had a much better goals against than Clemmensen (0.621 vs 1.058) but played against weaker competition as well (OppGF20 of .714 vs 0.743 for Clemmensen’s opponents).  Generally speaking Tomas Vokoun had a very weak 2007-08 season but much better 2008-09 and 2009-10 seasons even though he always seemed to play against weaker offensive opponents.

In terms of my Hockey Analysis Ratings, Tomas Vokoun ranked 16th out of 35 goalies in 2007-10 HARD and 18th in 2007-10 HARD+ rankings.  Middle of the pack.  The seasonal breakdown positioned him 35th of 38 in HARD+ for goalies with 500+ minutes in 2007-08, 19th of 35 in 2008-09, and 6th of 37 in 2009-10.  So far this season he is closer to the bottom again.

Is Tomas Vokoun an elite goalie, or even great goalie?  Probably not.  He just posts good save percentages because his team gives up a lot of shots, but not necessarily quality scoring chances, and he plays against weak offensive competition.

  27 Responses to “Tomas Vokoun: Elite or No?”

  1.  

    Vokoun’s backup stats are skewed because the team has historically played much better in front of backups. I haven’t noticed it as much with Clemmenson, but the team was notorious for playing responsibly in front of Andy while regularly hanging Vokoun out to dry.

    It’s disappointing to see his stock fall under more sophisticated statistical analysis, but half the argument loses its weight when this is taken into account.

  2.  

    Is there any evidence of this ‘hang him out to dry’ theory because in reality the opposite seems to be true. He gets played against weaker opposition and while he faced more shots per 20 minutes than Clemmensen last year he faced fewer shots per 20 minutes in 2007-08 and 2008-09 than Anderson.

    •  

      Unfortunately, the only evidence I have is second-hand anecdotal. I haven’t been able to watch many games, but we both know that even if I did it wouldn’t make a difference. I got the perception largely from other Panthers fans expressing frustration online with Vokoun letting in “weak” goals when he had no chance. It became a trend that they had unrealistic expectations, he’d get the blame when the trailer didn’t get picked up on a 3v1 or the defense left a loose puck from a rebound in the slot.

      Hardly compelling, but in the majority of the games I saw Anderson play, the team was far more responsible defensively. Sure, maybe he faced more shots, but I don’t think it’s inconceivable to think that the players had less faith in Anderson than Vokoun and stepped up their game for him in return.

  3.  

    David

    You miss the point here considerably.

    You rate a goaltender by his Fenwick number? Are you serious about that?

    Your argument that Vokoun plays against weaker opposition isn’t particularly accurate. He is the big game goalie for the Florida Panthers so he plays most of the games against divisional rivals. Florida plays in the weak Southeast Division. That tends to skew his apparant level of opposition downward (anyone who plays the southeast division a lot plays against a lower quality of opposition).

    For Florida a game against Tampa or Atlanta is a more important tougher game than one against Buffalo or Calgary – even if Buffalo or Calgary have more points that year.

    Vokoun face a lot of shots. His defence knows he can handle it and tends to play less conservatively when he is in net than his backup.

    I argue he is among the best goalies in hockey. I also argue that you started from the premise that he isn’t and looked for numbers to back that up while discarding any that showed otherwise.

    •  

      No, I don’t rate the goalie by his fenwick number. Where did I do that?

      The facts are facts. Vokoun plays against weaker offensive teams. Those are the facts.

      Vokoun face a lot of shots. His defence knows he can handle it and tends to play less conservatively when he is in net than his backup.

      Do you know for a fact that Vokoun faces more shots than his backups?

      2007-08 Vokoun: 11.027 shots against per 20 minutes
      2007-08 Anderson: 11.686

      2008-09 Vokoun: 11.758
      2008-09 Anderson: 12.916

      2009-10 Vokoun: 10.901
      2009-10 Clemmensen: 10.524

      Vokoun has faced less shots than Anderson and slightly more than Clemmensen so that doesn’t really support your theory.

      If in fact the Florida Panthers play differently with Vokoun in the net than his backups, they should stop doing so because they have more success playing with their backups in net.

      I argue he is among the best goalies in hockey.

      Yes, most people do, but if he was the best, wouldn’t you think he’d actually be able to win more games than he does?


      I also argue that you started from the premise that he isn’t and looked for numbers to back that up while discarding any that showed otherwise.

      I actually believed he was a very good goalie until I looked at how poorly he does at winning games and sought out an answer as to why. Believe me, I didn’t devise my hockey analysis rating system and create my stats.hockeyanalysis.com just to prove Vokoun isn’t an elite level goalie. It just worked out that way.

  4.  

    Do you think it is reasonable to rate a goalie 19th in the league with a 2.55/.925, and then 6th in the league the very next season with a .2.49/.926? Florida had the same coach, the same style of play and mostly the same roster, the only thing that changed was the backup goalie. Does playing with Clemmensen rather than playing with Anderson really make Vokoun the better goalie?

    If you are going to be comparing goalies to their backups, you should be looking at a multiple season sample. Even then you will still have certain goalies who have large advantages or disadvantages because of who they play with.

    For example, one could look at Nashville from 1999 to 2007, and note that Vokoun had a winning percentage of .503 while all other Preds goalies combined for .472. That’s a difference of 2.5 wins per season in an average of 47 starts, with an above-average level of teammates in Mike Dunham and Chris Mason.

    I’d also have to quibble a bit with your choice of metric for evaluating strength of opposition. If you are rating a goalie based on goal prevention, it doesn’t really matter how good their opponent is at outscoring, just how good they are at scoring. If you look at goals per game numbers for Vokoun’s opposition over the last three years compared to those of his backups, Vokoun does face slightly easier competition but the difference is very marginal.

    Related to that point, the Southeast Division typically has a lot of teams that can score goals but are even worse at preventing them. Since the lockout Vokoun is 2.74/.918 against the Southeast Division and 2.56/.923 against everyone else. I don’t see that as much of a statistical advantage for him at all to be honest.

    •  

      In 2009-10 he gave up goals at a pace of 0.621 per 20 minutes against opponents that scored at a rate of 0.714 per 20 minutes so he ‘slowed down’ his opponents production. In 2008-09 he gave up goals at a pace of 0.697 against opponents that scored at a rate of 0.683 per 20 minutes so his opponents did better against him. So it seems reasonable to me that he would be better rated in 2009-10 than 2008-09.

      The difference between Anderson and Clemmensen was non-existant. Anderson gave up goals at a rate of 0.986 vs opponents who scored at a rate of 0.695: 0.986/0.695 = 1.42. Clemmensen gave up goals at a rate of 1.058 against opponents who scored at a rate of .743. 1.058/0.743 = 1.42, identical to Anderson.

      Remember, I am just looking at 5v5 game tied situations. All other situations will influence a goalies gaa and save percentage. Vokoun must have played under a different set of circumstances in 2009-10 than 2008-09 for his numbers to even out. The purpose of the analysis is to factor out differences i playing situations not only to better compare goalies with each other but to compare a goalies performance season to season. As you point out, on the surface Vokoun’s last 2 seasons seem identical, but dig deeper and you can find differences.

      I’d also have to quibble a bit with your choice of metric for evaluating strength of opposition. If you are rating a goalie based on goal prevention, it doesn’t really matter how good their opponent is at outscoring, just how good they are at scoring.

      Yes, true. But, I was also interested in seeing why Vokoun had a poorer won-loss record so for this purpose looking at an opponents overall performance is necessary.

      Related to that point, the Southeast Division typically has a lot of teams that can score goals but are even worse at preventing them.

      Yes, but those go hand in hand. If they are bad at preventing goals the unbalanced schedule will artificially inflate their offensive numbers. This is why I consider my HAR?+ ratings better since it iteratively attempts to minimize these biases.

  5.  

    I don’t know why you would assume that there is no GAA effect just because the game is tied. That’s obviously a poor assumption, as your own stats show you that Vokoun faced 10.9 shots against per period at 5 on 5 with the game tied in 2009-10 compared to 11.7 SA/pd in 2008-09. Take that into account, and his save percentages at 5 on 5/tied are .943 and .941 respectively. With that kind of sample size there’s no significant difference between those two numbers, which is why I’d still question why he is rated much lower in 2008-09.

    Looking at the opponents’ overall record can provide some clues, but I think it’s more worthwhile to look at goal support with Vokoun in net compared to that of his backups. For example, in 2008-09 Florida averaged 3.30 goals per 60 minutes in games where Craig Anderson started and played the entire game, compared to 2.52 goals per 60 minutes in games where Vokoun started and played the entire game. I assume, like you do, that goalies have no impact on their team’s offence, which means it doesn’t appear to make much sense to blame Vokoun because his teammates scored at a much higher rate when he wasn’t in net.

    Re: the Southeast, I agree with you they are to some degree inflating their numbers against their own division, and overall the teams haven’t been very good, but the reality is that the Southeast still scored more against Vokoun since the lockout than the rest of the NHL did.

    •  

      I don’t know why you would assume that there is no GAA effect just because the game is tied.

      It would be far less significant than having a difference in the amount of PP or PK time the goalie experiences.

      That’s obviously a poor assumption, as your own stats show you that Vokoun faced 10.9 shots against per period at 5 on 5 with the game tied in 2009-10 compared to 11.7 SA/pd in 2008-09.

      Can you confirm for me that those 11.7 shots per period were of equivalent difficulty as the 10.9 shots per period? I don’t know. IMO, shots are driven by style of play, goals are driven by talent (i.e. shooting percentage). I know not everyone accepts this in the hockey statistical analysis world but it is what I believe and the evidence I have seen suggests that.

      Could my ratings be off? Could Vokoun be a bit better in 2008-09 than my analysis showed or a bit worse in 2009-10 than my analysis showed? Sure. My ratings are not perfect and certainly over just a single season there may be a margin of error that would be larger than ideal. That is why the three year analysis is probably better to look at. Either way, I’ll still claim that Vokoun is not an elite level goalie.

  6.  

    It would be far less significant than having a difference in the amount of PP or PK time the goalie experiences.

    Of course, but that’s not really related to my point, we’ve been talking about 5 on 5 play here. My point is that even if you want to assume that shot quality is the same across teams at even strength when the score is tied, which I think is a fair and supportable assumption, that does not mean that teams allow the same number of shots against. If Vokoun faced more shots in one year than another, it is only natural that his GAA also went up.

    Or are you instead claiming that the number of shots faced does not matter, all teams allow a similar number of scoring chances against per minute at even strength when the game is tied, and it is purely shooting and goaltending talent that makes all the difference?

    •  

      My point is that even if you want to assume that shot quality is the same across teams at even strength when the score is tied, which I think is a fair and supportable assumption, that does not mean that teams allow the same number of shots against.

      But I am not willing to make that assumption. I believe teams have different talent levels and scoring rates are more tied to shooting percentage than number of shots. So Vokoun may have faced more shots, but were they of equal quality? I am not willing to make that assumption, though I realize that a lot of people do.

      Or are you instead claiming that the number of shots faced does not matter, all teams allow a similar number of scoring chances against per minute at even strength when the game is tied, and it is purely shooting and goaltending talent that makes all the difference?

      Yes, this is closer to what I am saying but not exactly. Teams do take a different number of shots and number of shots does matter, but not as much as the quality of the shooter. 100 shots from the 2009-10 Maple Leafs and 100 shots from the 2009-10 Chicago Blackhawks or Washington Capitals are quite different. When it comes to scoring rates, shooting percentage drives them more than shot generation. See http://hockeyanalysis.com/2010/11/22/scoring-goals-shot-generation-or-shooting-percentage/

      Again, I am in the minority in this thinking as most people within the advanced hockey stats community feel perfectly OK in assuming that all shots are of equal quality and only number of shots (or fenwick/corsi) really matters. I have a hard time accepting that though.

  7.  

    I had a hard time accepting it too, until the evidence convinced me otherwise. But hopefully you can do a few follow-up posts on the topic, and if you can prove that the percentages are what is truly driving scoring then I’ll have to adjust my opinion.

    By the way, very nice stats site, love the breakdown by situation and score.

    •  

      I have seen the evidence too but I also know that all shots are not created equal. That is a fact. Plain and simple. I also know that teams can control shot totals as evidenced by scoring effects. Teams with the lead attempt to limit quality scoring chances, not shots. Teams with the lead often give up more shots. I believe assuming all shots are equal works to some extent because the league is filled with mass mediocrity. There are a lot of ordinary players (and teams) who are very much alike so of that large group, assuming all shots are equal can work reasonably well and probably better than looking at goals over smallish sample sizes. But it is still a flawed method based on an invalid assumption (all shots being equal) that we all know is invalid and it seems to fail miserably outside the mass mediocrity in the middle. Detroit didn’t have the 5 best forwards in hockey over the past 3 seasons and certainly Jonathan Cheechoo isn’t the 8th best player. To accept an analysis that produces that kind of result is nonsensical. It may work to some extent but it fails miserably under certain scenarios.

      IMO, what we really need to do when evaluating players is to use goals and use as large a sample size as possible so I would look at 3 years (or more) and 5v5tied (or even 5v5) data so a player such as Vokoun doesn’t get rewarded for playing against weak offensive players or for playing with a team that plays to limit quality chances, not shots.

  8.  

    I found evidence to suggest that shooting frequency is more important to winning than shooting percentage. This includes missed and blocked shots. Shot frequency has an R^2 value of .32 in 09/10 while shot percentage is only .182. When you calculate importance of shot frequency in a previous post you left out blocked and missed shots I believe, yet those are statistically significant variables. These results may skew your analysis in the quality of competition factor and therefore in your analysis of Vokoun. But I could be wrong, I have not done such a full analysis myself. Only computed shot frequency vs percentage.

    •  

      It is interesting because shot frequency does have a decent correlation with winning percentage, but it doesn’t have a great (relative to shooting percentage) correlation with scoring rates as shown in my previous article (where I used shots + missed shots). And that is what has me confused. If shooting percentage correlated highly with scoring rates (for and against) and scoring for/against correlates strongly with winning percentage, one would assume that shooting percentage should have a stronger correlation with winning than it does. Something is happening there that I don’t understand nor have I seen any one else attempt to explain. It probably just a result of randomness and small sample size.

      But does it make sense to evaluate using a metric (shots) that we know correlates with team wins but doesn’t seem to correlate well with scoring rates when evaluating individuals? Personally, I don’t think so, and as I explained above, the evidence is in the results. As good as Detroit is, they didn’t have that many top 5 players, and I am pretty certain Cheechoo isn’t that good as well. I believe my goal based HARD+, HARO+ and HART+ ratings (especially over a multi-year period) are a much more accurate indication of a players talent level than anything that could come out of a shot based method.

  9.  

    thats interesting… I did not compare shot frequency to score ratio, only wins. But i think the reason is simple. shot frequency may not contribute to as many goals as shot percentage, but what is does do i think is prevent goals against. More shots for one team generally means less shots for the other, due to puck control, and therefore less goals that team allows. Whereas a team with good shot percentage may not indicate how the other team is doing. A team with high shot frequency may not score more… but prevents the other team from scoring.

  10.  

    Additionally… Last season’s (09/10) save percentage had an R^2 value of about .156. I think what this means is that the goalies ability to have a high save percentage only contributes a small fraction to what determines a win and loss. I think one way to say it would be that the save percentage contributes to about 15-16% of the win/loss. And before anyway starts makign claims about one season being too small a sample size… keep in mind there are thousands and thousands of shots over the course of the season… if u compare every season to these values of save and shot percentage/frequency and R^2 values they vary very slightly. Not enough to be considered different.

  11.  

    Tomas Vokoun: Elite or No?

    Vokoun is the only thing holding the Cats together…
    If he had help in his own zone look out!

    Also, consider his Nashville numbers. All-Star.

    http://www.hockeyfanland.com

  12.  

    What evidence do you have that teams play differently at even strength when the score is tied?

    The observed correlation between EV tied shot differential and EV tied goal differential, over the last three seasons, is exactly what one would expect it to be if EV tied shot differential was the sole determinant of EV tied goal differential.

    In other words, the only reason that the observed correlation is lower than one is because of the limited sample size provided by a single NHL season. There is a method that can be used to correct for the sample size limitation by adjusting the observed correlation through taking into account the reliability coefficients of the involved variables. If this method is applied, the adjusted correlation between EV tied shot differential and EV tied goal differential is approximately 1.

    •  

      Why are you looking at shot/goal differential? Two teams can have similar differentials but different individual numbers. Look at shots for and shots against and goals for and against. We know that teams have the ability to change their style of play as evidenced by score effects, so why should we assume that all teams choose to play the same style at the same score? We know there are coaches/teams that stress defense first, while others stress puck possession while others are more offense oriented. Do the Washington Capitals play the same style as the New Jersey Devils? Do the rugged Flyers who hit a lot play the same style as the small Buffalo Sabres who don’t?

      •  

        “Why are you looking at shot/goal differential?”

        If teams do, in fact, play different styles when the score is tied at EV, then the “true” correlation between shot differential and goal differential — that is, the correlation that one would observe between the two variables over a sufficiently large sample — should be lower than 1.

        The fact that the true correlation is equal to or very close to 1 suggests that team to team differences in style of play at EV when the score is tied just aren’t that significant.

        And if you want me to demonstrate how I’m calculating the true correlation between the two variables, I’d be happy to do so.

        •  

          Maybe I am not understanding what you mean by shot differential and goal differential. By shot differential, do you mean shots for – shots against (or a ratio shots for / shots against or shots for / total shots)? My point is, a team that gets 10 shots for and gives up 10 shots against has the same shot differential as the team that gets 12 shots for and gives up 12 shots against, but the team giving/getting 10 shots might be playing a more defensive style by taking less risks offensively in order to take fewer risks defensively and hopefully fewer quality chances against.

          If you can show your math, that is always good since then we will assuredly understand where each other is coming from, but I still don’t think I’ll ever fully buy into the theory that we can assume all shots are created equal if we want to truly evaluate teams and players. It can serve as a decent proxy when dealing with small sample sizes, but given decent enough sample sizes I think goal analysis would provide more reliable results.

          •  

            By shot and goal differential, I was referring to (shots for – shots against) and (goals for – goals against). But everything I said would also apply equally to shot ratio and goal ratio.

            There were two reasons why I looked at shot differential as opposed to analyzing shots for and shots against separately.

            Firstly, because of recording bias – some rinks systematically overcount or undercount shots on goal. This problem is solved by looking at shot differential given that there’s no directional recording bias (that is, no arena appears to favor one team against the other).

            Secondly, at even strength in general, there is somewhat of a tradeoff between shot differential and PDO (shooting + save percentage), in that teams that do worse in terms of the percentages tend to do better in terms of outshooting, and vice versa. Moreover, the true correlation between shot differential and goal differential at EV is only about 0.8. Taken together, these findings imply the existence of team-to-team differences in style of play at even strength.

            However, if the same exercise is applied to team EV data with the score tied, one finds no correlation between shot differential and goal differential (and therefore no tradeoff) and a true correlation of approximately 1. This suggests that team differences in strategy at even strength are only relevant when the score margin is other than zero.

  13.  

    And in terms of calculating the true correlation…

    Because the reliability of both shot and goal differential over the course of an NHL regular season are imperfect, it’s necessary to adjust the seasonal correlation between the two variables to account for this.

    The adjustment involves dividing the observed correlation by the square root of the product of the reliability coefficients for each variable.

    Using data from the last three seasons, the reliability of EV shot differential when the score is tied is 0.7, and the reliability of EV goal differential when the score is tied is 0.43.

    In 09-10, the observed correlation between the two variables was 0.6. Thus, the true correlation is 1.09, given that [0.6/SQRT(0.7*0.43)]= 1.09. In 08-09, the observed correlation was 0.52, and the true correlation 0.95. And in 07-08, the observed correlation was 0.68, giving a true correlation of 1.24.

    •  

      Above you concluded that overall even strength teams seem to play different styles but when game tied they all appear to play the same style. Have you looked at leading and trailing scenarios to determine whether teams play differently when say down 1 or do all teams play the same when down 1 (which is different than tied). That is to say, do all teams play the same style as each other under each particular scenario and the difference we see in overall even strength is due primarily to differences in time played under the different score scenarios (i.e. some teams will play more time leading than trailing).

      •  

        I was actually about to throw up a post on that very subject.

        The differences persist even when controlling for goal state.

        Also, going back to our original discussion, I will concede that it’s possible that teams do play differently with the score tied, but that the style of play differences do not produce a tradeoff between shot differential and PDO, as is the case in general.

        For example, it’s conceivable that a more conservative style of play would have a negative effect on both shooting and save percentage, as well as both shots for and shots against, thereby leaving both shot ratio as well as PDO unaffected.

        If that was your original point then I apologize, as that could very well be the case. I’m just not sure if there’s any supporting evidence (sample size limitations and recording bias complicate things here).

Sorry, the comment form is closed at this time.