So if shot quality exists in the long term, team-wide sense, we’d expect to see certain teams shoot at a relatively higher rate over the long term and multiple seasons and teams that lack the ability to generate high quality chances have lower shooting percentages. But that’s not the case:
http://canucksarmy.com/2011/11/9/can-te ... lity-shots
To really see if shot quality matters offensively, we’d want to look at the entire league, over a period of years. Thanks to Behind the Net, we can do that – we have four years of 5-on-5 shooting data, from 2007-08 to 2010-11. We’ll run mathematical correlations, to see the relationship from one year to the next – a score of 1 represents a perfect correlation, a score of zero shows no correlation whatsoever.
2007-08 to 2008-09 correlation: 0.179
2008-09 to 2009-10 correlation: -0.067
2009-10 to 2010-11 correlation: -0.121
Average year-to-year correlation: -0.003
The average correlation is actually slightly negative over these years, but it’s very, very close to exactly zero. In other words, there seems to be no connection between how good a team’s shooting percentage is from one year to the next. This is a significant argument that there is no major difference between individual NHL teams in their ability to score on any given shot – over the big picture, shot quality in 5-on-5 situations evens out.
But you know what does have some correlation year over year: shot generation.
While a team’s shot quality seems to bounce around erratically from year to year, the same is not true of the number of shots that they take. Here are the same correlations, but this time instead of looking at team shooting percentage, we will look at team shooting rates (shots/60) in 5-on-5 situations:
2007-08 to 2008-09 correlation: 0.578
2008-09 to 2009-10 correlation: 0.453
2009-10 to 2010-11 correlation: 0.462
Average year-to-year correlation: 0.498
That’s not a perfect correlation by any means, but there’s clearly a relationship between how teams perform from one year to the next – something we didn’t find when we looked at shot quality.
Why? If a team is capable of generating higher shot quality chances, why is there no connection between a team’s shooting percentage one year to the next?
Because the real driver is the ability to maintain puck possession and generate shots in general. More shots leads to more quality chances leads to more goals:
http://www.hockeybuzz.com/blog/Travis-Y ... ZBjHLUp-So
“The overarching point: a game can be stolen by a goaltender, and a game can be stolen by forwards shooting at a ridiculous percentage clip. But, over time, luck will wash it all out. And, shot quality -- one of the last bastions of old-time hockey -- simply doesn't play the role many wrongfully assumed, self-included, in producing consistent, winning hockey. “
mikey287 wrote:We really still think save percentage isn't a team stat at this point with modern defending...? Yeesh...
Brodeur sucks, Rask is the greatest goaltender in history and Ottawa has three of the eight best goalies in hockey...I'm sold.
It's such a beautiful game...
On the flip side, if “team defensive systems” are capable of influencing shot quality to the point that save percentage is irrelevant to the point that you don’t care about it as you told me, we’d expect that when a goalie plays Season 1 with Team X and Season 2 with Team Y, there should some pretty significant swings in save%, right? As you said, save% is a team-specific stat – which, by the way, makes it all the more hilarious that you would reject save% as it relates to Reimer of all people, who is playing behind one of the worst defensive corps in the NHL. You specifically pointed to Bryzgalov here, and the history of Boston goaltenders as supposed evidence that goalies switching teams or arriving on teams with defensive systems that suppress shot quality as “proof” that save% isn’t a very valuable metric for evaluating goalies, so I would assume you’d agree that looking at year-over-year goalie moves from team to team should show significant changes.
Guess what, they don’t at all:
http://vhockey.blogspot.com/2009/07/sho ... ntasy.html
The thinking was that if teams were the ones impacting the save percentage, it would reveal itself when the goalies switched squads. I'm giving the same idea a more rigorous test here.
And the universe [he means random; he built a model based on coin flips and ran it 100 times per goalie who qualified] requires that the EVsave% change as the goalie moves from team to team, if shot quality doesn't exist at all, well it will average .0122. With these 66 goalies it averages .0133. There is nowhere near enough veracity to even declare that a difference in shot quality exists at all, but again the distribution is a smidge wider with the real than it is with the random, so I'll give the benefit of the doubt and say that, on average, the difference in shot quality from any two teams selected at random ... the expected difference will be 2 goals on the season.
If shot quality exists such that it influences save%, why doesn’t this show up in the 66 goalies who qualified [min. 300 EV shots against per season]? The difference between a completely random expected change in save% from year to year and what actually happened is nearly identical. Please explain how that can be if save% is a “team-based” statistic? Citing a single instance – Bryzgalov – is meaningless (and I haven’t looked at how PHX v. PHI plays in terms of S/A, times shorthanded, etc., plus soft factors, like Bryz is a headcase who thrived in a low-pressure hockey environment but broke down in a place that eats goalies alive like PHI).
More specifically as it relates to save%:
The average change in a goalie's save percentage from year to year when he stays with the same team is just 0.0005 larger than simple random chance would predict, and when a goalie changes teams the sv% difference is just 0.0011 larger than random chance. The best shot-quality-influencing system of this era (Jacques Lemaire's) reduced Fenwick shooting percentages by about 0.0015. The result is that any team effect on a goalie's save percentage doesn't add up to more than a goal or two per season.
The point is not: "All teams face the same shots, so ignore shot quality completely."
The point is: "Differences in shot locations are small and require a very large data set to overcome noise, so you won't be wrong by much if you ignore them."
http://nhlnumbers.com/2012/7/3/shot-qua ... t-how-much
So yes, you very much should start paying attention to a goalie's save% in evaluating them. Good team defensive system's reduce shots against as a whole; limit special teams opportunities; and drive possession themselves by possessing the puck and taking more shots than the other team (and having a good goalie who bails them because he's talented and makes a lot of saves -- and as a result, horror of horrors, posts a high save%).
The sum total is that shot quality exists on a shot to shot basis, and certain players generate higher quality chances because they are talented enough to generate shots in the first instance. Individual players can drive shooting percentages, but its not big enough that its driving team outcomes and/or save%. Most peg shot quality as maybe 5-10 percent of outcomes in the long term.
You’ll no doubt dismiss this as “garbage in, garbage out” or some other such nonsense because it challenges your pre-conceived notions about the game. If you’d pay attention, you’d notice just how rigorous and valuable this information is.
mikey287 wrote:Stat community: "We don't want human subjectivity to get in the way of player evaluation...so, we'll use these statistics based on human subjectivity from humans that may or may not know even less about the game than we do to eliminate this subjectivity..."
I mean, how does shot quality not exist? Because it can't be quantified, therefore it's attacked? That seems too convenient.
The real issue here is that quote above. You seem to think that people who “like” statistical analysis somehow believe that the “traditional” means of player evaluation are useless and should be discarded. You completely invented that part of your post; I never said that. Its a complete and utter strawman and you shouldn't have written it.
Shot quality can be quantified -- imperfectly sure, but we're at a point that we have enough data and enough independant charting of scoring chances and shot charts that this isn't the "crap" you're disingenuously making it out to be.
The people building these fantastically complex models and thinking creatively about how to utilize the data generated by a hockey game don’t think that human analysis should be removed either. I can look at a player’s Corsi or QualComp or Usage Chart as a way to tell me about that player and his value; it doesn’t tell me WHY he is good/bad according to analytics. Maybe it’s his skating ability; maybe it’s positioning and hockey sense; maybe its passing. I can look at player X and have a decent idea of his value, but I can't provide the same expertise about the nuts and bolts of why like you can. But to pretend that “stats” are this boogeyman; man, you are being left behind. It happened in baseball in the 2000’s and its happening in hockey now. Its the exact same thing I was reading on FJM then; pre-conceived notions challenged ---> traditional thinkers get very nervous, say things like DERP DO YOU EVEN WATCH THE GAMES YOU LIVE IN YOUR MOTHERS BASEMENT YOU WANT THEM PLAYED BY ROBOTS YOU DONT WANT THE HUMAN ELEMENT -----> eventually everyone comes to their sense and realizes that like every other highly competitive industry, information -- and the ability to process it effectively -- is power. John Davidson(!) hired a GM who said this:
'Stats are facts...In the long run they hardly ever lie. Thorough analytical work, like that done by Hockey Prospectus, is needed to make a proper evaluation of them."
- Jarmo Kekalainen
There's no "magic stat" that I can give you to tell you who is the "best" goalie or center or coach. What I can give you is a bunch of data generated by the actual players playing the actual games that reflects their performance, and that is processed to take into account many, many variables like competition, context, special teams, linemates, score effects, home arena bias, etc. Combined with experience and knowledge, that's a pretty good picture of a player's ability.
There’s a lot more to address your posts (Reimer in particular) and I will eventually, but this is a good starting point. If you disagree, try to do better than "because I say so." I am genuinely interested.