Efficiency Snapshots
Recently I posted some game-by-game adjusted efficiency ratings for Kansas, derived from Ken Pomeroy’s Game Plan and season efficiency ratings. The Hawks’ numbers looked good, but Jeremy asked for some context on how the numbers were changing as the season progressed, and how this compared to other top teams. So I ran the game-by-game numbers for Pomeroy’s top 11 teams. (Why top 11? I’ll explain Michigan State’s case later on.) Just showing you a mess o’ single game numbers doesn’t do a whole lot of good - there’s a lot of game to game variation. To smooth that noise out and get a better idea of a team’s general trend, we can look at a moving 10-game snapshot.
About the graphs: Each team’s line starts with its 10th game and continues through its most recent (as of Wednesday afternoon, so OSU’s stinker vs. Penn St is NOT included). Each point is the average of the 10 previous games. The X-axis is “games ago.” I tried to get the line colors to mostly correspond to school colors, but there are sooooo many schools that use blue or red. Anyway, here you go…
I really like the look of this one. You can see that for the first half of the season, KU’s offense wasn’t at the level of the other elite teams, but over the last month it’s steadily risen.
Georgetown’s curve looks similar, only they start higher and finish in uber-elite territory. Interestingly, every single team on here has improved over the course of the season. I’m wondering if that’s all selection bias (we’re looking at the best teams as of NOW, so obviously the recent ratings will be high), or if it’s also partly due to the fact that offense is just more difficult to perfect than defense. So defenses start the year already performing at a high level, and the offenses catch up over the next few months.
[EDIT: Please note that in this graph, DOWN IS GOOD!]
This looks suspiciously like a jumbled mess. Picking out KU’s line, you can see they’ve bounced around between 80 and 85 the whole year, always maintaining their spot as one of the top teams. For a while North Carolina seemed to be quite a bit better than everyone else, but they’ve fallen back to only “great.” (I’m betting the same thing happens with Georgetown’s offense over the next few weeks. [EDIT: I originally made a typo and said GTown’s defense. This caused some confusion over on Hoya Talk. My bad.]) You can see that this graph doesn’t show the consistent improvement that the offenses do.
[WARNING: If you couldn’t care less about Michigan State, skip this paragraph.] I think MSU’s path is the most interesting one here, and is the reason they’re going to be really surprising some people over the next month. Their defense went from elite, to just better than average, back to elite. I took a look a closer look to see if there were injuries that could explain this, and it turns out they were missing freshman Raymar Morgan for most of that swoon. Judging from his scouting report, that didn’t seem like such a huge loss. He’s a subpar offensive player, and most of his playing time was taken up by another 6 1/2 foot freshman, Isaiah Dahlman. Problem is, Dahlman’s only an inch shorter but 40 pounds lighter. He doesn’t rebound, block shots, or steal the ball as well as Morgan, and I’m guessing he’s easier to score on. Dahmlan played at least 18 minutes in 8 games this season, mostly while Morgan was out. In those 8 games, Michigan State’s adjusted defensive rating was 93.5. In all other games, it’s 83.4. I wish I would have noticed this 2 days ago, so I could feel smart for predicting a MSU victory over Wisconsin.
Since around the first week of January, North Carolina has, from an efficiency standpoint, looked like the team to beat. Wisconsin approached their level for a while, as did Texas A&M, Ohio St, and Florida, but nobody else had managed to crack the 0.99 barrier, while UNC had been staying comfortably above it. Well, congrats to Kansas on joining them up there. This is a nice looking graph for Kansas, showing that they seem to be putting it all together. Only problem is, most of this nice rating has come from beating up on lesser opponents. Not bad opponents, necessarily, but lesser. They let up and gave Acie Law IV the win in Lawrence in their one chance to prove they could play elite ball against an elite opponent. Still, the stats are what they are, and they make Kansas look good.
OK, I’ve got nothing more to add right this moment. I’ll probably be doing some kind of individual team graphs for KU game previews in the future. If anybody has any ideas on ways to slice these numbers, or different graphical displays that you think might be interesting or useful, feel free to mention them.
* Have the Phog Blog come to you! * Subscribe to Comments
15 Responses to “Efficiency Snapshots”
Leave a Reply
Comment Policy:Be polite.You must be logged in to post a comment.
February 22nd, 2007 at 5:15 pm
This is awesome. Thanks DavidH!
February 22nd, 2007 at 5:27 pm
Incredible job David. Love the insight and the graphs really help in the analysis. I still haven’t seen Georgetown play this year. Is there Offense really that strong? Michigan State’s path truly is intriguing. Imagine Self, Roy and Coach K all having good defensive teams (the only 3 teams whose defensive efficency haven’t gone above 86).
February 22nd, 2007 at 5:34 pm
*…at least in the last 15 or so games. I nearly went blind trying to follow the North Carolina line.
February 22nd, 2007 at 6:46 pm
Great work David… hopefully the data is easy to plug into your system?
As to the adjustment itself, I tried to post a response in our conversation on this some time ago and it didn’t work. It is currently here:
(it may need word wrapping)
http://www.freefilehosting.org/public/38022/adjustment.txt
I think anybody who watches basketball this year could tell you that Duke isn’t a great defensive team, Game Plan would probably back that up, and yet they are 3rd in the ranking.
February 23rd, 2007 at 11:07 am
Chalmersfan,
FIRST, yeah, it’s pretty easy. Only thing that takes time is the graphs. Hopefully I can figure out a faster way to do those. But on to the meat of this comment…
SECOND, about your wariness of adjusted data, specifically the Pac-10 defenses - I’m not sure looking at conference results from a single conference can tell you much at all about whether the adjustments make sense. All you can see is that the P10 has more points per possession than average. That could result from any of these things:
A) average D + good O
B) bad D + average O
C) good D + REALLY good O
D) REALLY bad D + bad O
I think you need to look at nonconference games to figure out how they should be adjusted relative the whole country. Take UCLA, for example. You mentioned that half their non-con games were lame, but half were most definitely not. Take a look at these opponents, along with their raw and adjusted defensive ranks, and UCLA’s raw efficiency against them:
Against good defenses, they averaged a 115.8 OffEff.
Now look at the Pac-10. I’ll split into up into two groups - the ones with bad raw defense numbers, and the ones with good raw defense numbers.
The good Pac-10 defenses, despite ranking far below the non-conf opponents in RAW defense, outperformed them against UCLA. They were adjusted up quite a bit, but still not up to the level of the non-conf opponents.
The BAD Pac-10 defenses obviously didn’t do as well as the good non-conf opponents. But they didn’t do horribly worse (120.0 vs. 115.8). Look at the equivalent defensive efficiency for the pre- and post-adjustment rankings of the non-conf and bad P10 opponents:
The difference between their ratings post-adjustment is closer to the actual difference in performance. Now, obviously this is just one team, and only some of their games. I’m not going to go through and do this for every team, but I suspect you’d find similar results as a whole.
Did this make sense? I feel like I might be totally missing the point or making some kind of circular argument. Anyway, it makes sense to me.
LASTLY, about Duke:
Yeah, Duke is ranked 3rd in adjusted defense, but it’s not because they’re getting some kind of monster adjustment. They’re already ranked 6th in raw. And they’ve done it against one of the toughest schedules in the country (Pomeroy has their schedule ranked 4th, RPI has it 3rd, Sagarin has it 7th, etc.). Here’s what Game Plan tells me - they’ve played 28 games this year, and held their opponent below their average offensive efficiency (both raw and adjusted) in 27 of them. The one bad defensive game was in November against Marquette, and even that one turns out to be right at average for Marquette’s adjusted efficiency.
Sorry if it seems like I’m trying to be argumentative.
February 23rd, 2007 at 11:23 am
OK, I just checked out KU, UNC, Texas A&M, and Florida on Game Plan to see how Duke’s results compare.
KU - 1 average game (DePaul)
UNC - 2 bad (NC St, VTech), 2 average (High Point, Ohio St)
TEXAS A&M - 1 bad (Tex Tech), 4 average (Tex Tech, Fordham, LSU, LA Tech)
FLORIDA - 3 bad (Vandy, UAB, Stetson), 3 average (Tenn, FSU, KU)
So it looks like maybe Duke is keeping their high rating due to their consistency. Perhaps what you’re seeing to make you disagree with the rating is a lack of ability to completely shut down a team. But what the rating sees that maybe makes up for it is extreme consistency - they just don’t lay any eggs defensively.
February 23rd, 2007 at 1:21 pm
I also look at Pomeroy’s rankings a lot, but the one thing that scares me about the top two teams (UNC & KU) in his ratings is there consistency. Just from watching games and seeing scores it seems like these two teams have been fairly inconsistent, but I decided to go ahead and try to quantify it myself. Using the point spreads for each team’s conference games, I took the numbers of how far the final score in the game was from the expectation (the spread). I then took the standard deviation of these numbers, to theoretically come up with a consistency ranking for the top 40 teams in Pomeroy’s ratings (as of last night, Butler & Xavier excluded as I don’t have their #s). Here’s what I came up with (Column 2 is Pomeroy rank, column 3 is their standard deviation, and column 4 is their consistency rank, with 1 being the most consistent. The new order of the teams is my attempt to incorporate consistency and potential, giving 90% weight to Pomeroy’s rating, and 10% weight to my consistency rating):
[EDITOR’S NOTE: I reformatted this post. Hopefully the table shows up now. If not I’ll change it back.]
As expected, Kansas is the 37th most consistent out of the 38 teams (until the K State game had covered the spread 13 times, none by less than 8 points). In the tournament, it is theoretically good to be more consistent if you are better, as to sustain a high level of play throughout, and less consistent if you are the inferior team, giving you the opportunity to play over your head on a certain night and pull an upset. The next step in this analysis would be to take Lunardi’s seeding of each of these teams, figure out the average ASM differential between each seed, and see what % each’s teams chances are of advancing to each subsequent round. I’ll work on that.
February 23rd, 2007 at 2:03 pm
Without thinking on this too long, Jacob, I have to ask: are you sure that what your results measure isn’t some combination of inconsistency and the ability to blow out bad teams?
Kansas is +25 per 100 possession in league play. UNC is +20. Florida is +18. Memphis is +30. In those cases, almost all of the inconsistency would still tend to describe how much they won by rather than whether or not they won which is certainly an issue for teams like Virginia Tech, Missouri State, Maryland, or Purdue.
February 23rd, 2007 at 2:12 pm
I agree. The theory behind the efficiency rating is that these top tier teams have already gotten credit for these blowout wins through the Pomeroy system. And I am not just taking how much teams win by, which would unfairly penalize teams for winning by a lot. Say for instance Kansas was favored by 20 (in both Vegas and Pomeroy- I am going to try to start using Pomeroy’s predictions rather than the spread). If they win by 35, they will get credit for winning by 35 in Pomeroy, and thus their ranking will be significantly impoved. They will also lose a little ground in the consistency rankings, since they were 15 off the prediction. So this blowout win will obviously help KU. The team they played, on the other hand, would go down in both Pomeroy and consistency.
February 23rd, 2007 at 2:13 pm
I like the idea of adjusting for consistency, but I think that doing so using some factor based on how they performed relative to expectation would be useful.
That way you could map upward and downward stability.
Asteroid, who sometimes posts on PB, used to account for trend relative to expectation, but I wasn’t sure exactly how he did it.
February 23rd, 2007 at 2:14 pm
That was a little long winded, but here’s my point: if you play a team you’re theoretically 20 points better than, Pomeroy would give you (roughly) the same credit for winning the two games by 35 and 5 as winning both games by 20, while in the Tournament I would rather have the team that won by 20 both times.
February 23rd, 2007 at 2:48 pm
Rock Chalk Spreadsheet…
The good folks (or is it folk?) at the Phog Blog paid a little visit to Ken Pomeroy’s stats, fired up Excel, tippy-tapped on a few keys and cranked out a few nifty graphs that are worthy of your perusal…….
February 24th, 2007 at 12:10 am
Ok David, here’s what I’ve gotten from your post… I’ll reason my way through it…
When UCLA played 6 good noncon teams collectively giving up an OE of about 96 their OE was 116.
When UCLA played 9 bad Pac 10 defenses giving up an OE of about 104 their OE was 120.
When UCLA played 6 good Pac 10 defenses giving up an OE of about 96 their OE was 104.
How do I interpret this? When they faced good defenses their offense was very good. When they faced bad defenses their offense was great and then when they faced some more good defenses their offense was just slightly above average.
Looking at their season it seems that their offense started hot, cooled down a lot and is now on a roll the last 3 games.
What I can’t wrap my mind around is why the adjustment in their OE is what it is- namely, bigger than KU’s, especially when according to the raw ranks on this page (http://kenpom.com/stats.php?y=2007&s=18) Big 12 teams are better on defense. I may just have to accept that UCLA’s non-con and overall schedule was/is harder than KU’s and leave it at that. It seems too circular to me and in my gut I stil have doubts about season-long adjustment especially when it seems like teams in Pomeroy-strong leagues get the best adjustments even when the teams in those leagues have weak ranks for their raw categories.
However, I’m not sure any further argument or presenting of stats can solve this… it may just be me.
February 24th, 2007 at 6:49 pm
DavidH: really interesting stuff, thanks for posting. It would be interesting if you could show an ‘average’ line in there somewhere.
ChalmersFan: if you’ve watched Duke at all this year, you know the only reason we’re above .500 (hell, the only reason we’re above about .100) is because the defense is excellent. Georgetown, Air Force, and Carolina are (unadjusted) the #2,3, and 4 teams in offensive efficiency this year (per Kenpom), with raw numbers around 117. We held them to efficiencies of 87.3, 100.2, and 105.6, respectively. If holding three of the best four offenses in the country substantially under their efficiency averages isn’t a good demonstration of good defense, you’re unconvincable
(You can go through the rest of the efficiency rating and Duke’s schedule if you’re not convinced; Duke very regularly makes very good offenses look mediocre and very bad defenses look good this year; BC and Gonzaga are other games and stats to look at.)
February 24th, 2007 at 8:23 pm
Average is 101.8 for offensive and defensive efficiency, and .5 for Pythagorean rating