Bracket Breakdown, Bradley-Terry Style
by SpartanDan on Mar 17, 2010 7:39 AM CDT
Email
Print[Bumped. Another tool for anyone who, like me, hasn't filled out their bracket yet. Dan, can you provide a little more technical information on how you calculate these ratings? Looks like a ton of work. -KJ]
(Alternate title: This Is Your Bracket on KRACH.)
Undoubtedly you have seen the log5 breakdowns of the bracket over at Basketball Prospectus, based on Ken Pomeroy's rankings. (If you haven't, what's stopping you? The link is to the Midwest preview, but the others are available as well.) You may also recall that back in December, I posted a short article on the Bradley-Terry ranking method (familiar to college hockey fans as KRACH) and applying it to college basketball as a potential RPI substitute. The main point in its favor is that exceptionally strong or weak opponents cannot have such a significant effect on strength of schedule that a loss raises your rating or a win lowers it; this is a significant flaw in the RPI (and some other record-only rating systems; it can happen in points-based rankings like Pomeroy's but that's because you won by less than his model predicted, so it's not really a flaw there). In fact, if you were to play the world's most awful team, a team that would never beat anyone, your rating would remain exactly unchanged (assuming you won). Likewise if you lost to a theoretically perfect team who had an infinite rating.
Another useful point about the Bradley-Terry method is that it, like Pomeroy's rankings, provides an easy method to calculate the odds of one team beating another: simply divide Team A's rating by the sum of (Team A's rating) and (Team B's rating), and you have Team A's probability of defeating Team B. (The formula for Pomeroy's rankings is only slightly more complex.) So I decided to do a similar bracket breakdown, giving odds of each and every team reaching each round.
Odds table after the jump.
| Team | Seed | B-T Rating | 2nd Round | Sweet 16 | Elite 8 | Final 4 | Title Game | Champ |
|---|---|---|---|---|---|---|---|---|
| Kansas | M1 | 95.3024 | 99.24% | 87.98% | 78.51% | 67.24% | 52.22% | 38.24% |
| Kentucky | E1 | 59.9940 | 98.36% | 82.45% | 63.84% | 44.67% | 32.60% | 17.42% |
| Syracuse | W1 | 43.7536 | 96.85% | 77.40% | 59.79% | 42.18% | 19.15% | 10.84% |
| West Virginia | E2 | 35.4880 | 95.98% | 75.83% | 50.67% | 24.29% | 15.01% | 6.30% |
| Duke | S1 | 29.6715 | 98.71% | 76.44% | 48.07% | 29.81% | 13.36% | 5.11% |
| Villanova | S2 | 24.3322 | 96.00% | 66.38% | 41.52% | 21.70% | 8.74% | 2.99% |
| Kansas St | W2 | 24.8326 | 93.90% | 61.30% | 40.01% | 19.26% | 6.59% | 2.86% |
| New Mexico | E3 | 23.2448 | 91.93% | 64.65% | 30.60% | 11.80% | 6.10% | 2.03% |
| Purdue | S4 | 23.8606 | 77.48% | 50.70% | 26.05% | 14.83% | 5.91% | 2.00% |
| Temple | E5 | 24.3970 | 71.62% | 49.37% | 17.60% | 8.74% | 4.62% | 1.59% |
| Georgetown | M3 | 20.1088 | 91.46% | 54.82% | 33.83% | 9.15% | 4.07% | 1.57% |
| Baylor | S3 | 19.3236 | 90.74% | 62.27% | 31.76% | 14.84% | 5.23% | 1.56% |
| PIttsburgh | W3 | 17.2214 | 82.78% | 52.88% | 24.71% | 9.87% | 2.74% | 0.96% |
| Ohio St | M2 | 16.2168 | 88.81% | 57.34% | 28.18% | 6.68% | 2.64% | 0.89% |
| Butler | W5 | 16.7066 | 65.70% | 40.92% | 14.37% | 6.96% | 1.89% | 0.65% |
| BYU | W7 | 16.4780 | 68.24% | 29.35% | 16.39% | 6.39% | 1.73% | 0.59% |
| Tennessee | M6 | 16.4854 | 64.64% | 31.59% | 18.03% | 4.32% | 1.72% | 0.59% |
| Texas A&M | S5 | 15.8677 | 70.84% | 33.02% | 13.84% | 6.52% | 2.03% | 0.54% |
| Maryland | M4 | 12.9482 | 86.71% | 50.72% | 8.46% | 3.86% | 1.34% | 0.39% |
| Vanderbilt | W4 | 12.8231 | 71.64% | 35.07% | 10.46% | 4.40% | 1.01% | 0.29% |
| Wisconsin | E4 | 13.6242 | 79.52% | 33.61% | 8.30% | 3.01% | 1.17% | 0.28% |
| Xavier | W6 | 12.1174 | 67.33% | 31.97% | 12.29% | 4.01% | 0.89% | 0.25% |
| Michigan St | M5 | 11.7461 | 76.19% | 40.09% | 6.22% | 2.69% | 0.88% | 0.24% |
| N Iowa | M9 | 14.0160 | 59.14% | 7.95% | 4.43% | 2.10% | 0.76% | 0.24% |
| Texas | E8 | 13.6891 | 62.03% | 12.28% | 5.56% | 2.02% | 0.79% | 0.19% |
| Gonzaga | W8 | 11.8179 | 54.00% | 12.64% | 6.14% | 2.47% | 0.54% | 0.15% |
| St Mary's | S10 | 11.2242 | 51.73% | 17.58% | 7.79% | 2.67% | 0.66% | 0.14% |
| Marquette | E6 | 11.5208 | 60.63% | 22.63% | 7.25% | 1.81% | 0.64% | 0.14% |
| Notre Dame | S6 | 10.3006 | 59.91% | 23.56% | 8.54% | 2.77% | 0.64% | 0.13% |
| Richmond | S7 | 10.4714 | 48.27% | 15.70% | 6.71% | 2.20% | 0.52% | 0.10% |
| Oklahoma St | M7 | 9.7051 | 54.87% | 23.32% | 8.61% | 1.45% | 0.42% | 0.10% |
| Florida St | W9 | 10.0687 | 46.00% | 9.60% | 4.29% | 1.57% | 0.31% | 0.08% |
| Missouri | E10 | 9.6902 | 51.40% | 12.37% | 4.57% | 1.01% | 0.32% | 0.06% |
| Louisville | S9 | 8.9170 | 51.61% | 12.41% | 4.34% | 1.47% | 0.31% | 0.05% |
| San Diego St | M11 | 9.0186 | 35.36% | 12.51% | 5.33% | 0.85% | 0.23% | 0.05% |
| UNLV | M8 | 9.6847 | 40.86% | 4.03% | 1.88% | 0.73% | 0.21% | 0.05% |
| UTEP | W12 | 8.7206 | 34.30% | 16.09% | 3.70% | 1.23% | 0.22% | 0.05% |
| Clemson | E7 | 9.1622 | 48.60% | 11.25% | 4.02% | 0.85% | 0.26% | 0.05% |
| Cornell | E12 | 9.6673 | 28.38% | 13.63% | 2.64% | 0.77% | 0.24% | 0.05% |
| Georgia Tech | M10 | 7.9837 | 45.13% | 17.25% | 5.63% | 0.82% | 0.21% | 0.04% |
| California | S8 | 8.3612 | 48.39% | 11.10% | 3.72% | 1.21% | 0.24% | 0.04% |
| Florida | W10 | 7.6704 | 31.76% | 8.64% | 3.27% | 0.79% | 0.13% | 0.03% |
| Wake Forest | E9 | 8.3804 | 37.97% | 5.13% | 1.75% | 0.46% | 0.13% | 0.02% |
| Old Dominion | S11 | 6.8942 | 40.09% | 12.45% | 3.47% | 0.86% | 0.15% | 0.02% |
| Washington | E11 | 7.4802 | 39.37% | 11.31% | 2.73% | 0.50% | 0.13% | 0.02% |
| Siena | S13 | 6.9356 | 22.52% | 8.23% | 2.06% | 0.59% | 0.10% | 0.01% |
| Utah St | S12 | 6.5318 | 29.16% | 8.04% | 1.93% | 0.53% | 0.09% | 0.01% |
| Minnesota | W11 | 5.8808 | 32.67% | 10.38% | 2.47% | 0.49% | 0.06% | 0.01% |
| Murray St | W13 | 5.0753 | 28.36% | 7.92% | 1.21% | 0.28% | 0.03% | 0.005% |
| New Mexico St | M12 | 3.6712 | 23.81% | 6.62% | 0.40% | 0.08% | 0.01% | 0.001% |
| Oakland | W14 | 3.5825 | 17.22% | 4.78% | 0.77% | 0.10% | 0.009% | 0.001% |
| Wofford | E13 | 3.5096 | 20.48% | 3.39% | 0.29% | 0.04% | 0.006% | 0.0005% |
| Santa Barbara | M15 | 2.0441 | 11.19% | 2.10% | 0.24% | 0.01% | 0.001% | 0.00006% |
| Houston | M13 | 1.9848 | 13.29% | 2.57% | 0.09% | 0.01% | 0.0009% | 0.00005% |
| Sam Houston St | S14 | 1.9723 | 9.26% | 1.72% | 0.18% | 0.02% | 0.0009% | 0.00005% |
| Montana | E14 | 2.0411 | 8.07% | 1.42% | 0.12% | 0.008% | 0.0007% | 0.00004% |
| Ohio | M14 | 1.8769 | 8.54% | 1.08% | 0.15% | 0.007% | 0.0005% | 0.00003% |
| North Texas | W15 | 1.6135 | 6.10% | 0.71% | 0.09% | 0.006% | 0.0003% | 0.00001% |
| Morgan St | E15 | 1.4848 | 4.02% | 0.55% | 0.05% | 0.002% | 0.0002% | 0.000006% |
| Vermont | W16 | 1.4211 | 3.15% | 0.36% | 0.04% | 0.003% | 0.0001% | 0.000006% |
| Robert Morris | S15 | 1.0140 | 4.00% | 0.34% | 0.03% | 0.001% | 0.00004% | 0.000001% |
| East Tenn St | E16 | 1.0015 | 1.64% | 0.14% | 0.009% | 0.0004% | 0.00002% | 0.0000005% |
| Lehigh | M16 | 0.7255 | 0.76% | 0.04% | 0.003% | 0.0001% | 0.000004% | 0.0000001% |
| Ark Pine Bluf | S16 | 0.3863 | 1.29% | 0.06% | 0.001% | 0.00003% | 0.0000004% | 0.000000004% |
Blue indicates a team that the Bradley-Terry method rates more highly than Pomeroy's (in terms of probability of beating an "average" team), red indicates the reverse. The darker the shade, the more significant the difference. The most extreme positive difference: New Mexico is expected to win 95.88% of the time against an average team by the Bradley-Terry method but just 88.21% by Pomeroy. Utah State, on the other extreme, is expected to win 86.72% by the Bradley-Terry method but 93.01% by Pomeroy. (Minnesota and Wisconsin, interestingly, are the two next most extreme in B-T pessimism. Oakland and Kentucky are second and third in B-T optimism.)
Some notable differences, bullet-style:
- Kansas is #1 by a mile in the Bradley-Terry rankings; only Kentucky and Syracuse even have a 30% chance of pulling the upset. This and the models' disagreement on Ohio State and Maryland (Pomeroy likes both considerably better) leads to a big jump in Kansas's title chances.
- Bradley-Terry gives Duke just over a 5% chance to win it all, compared to 24% by Pomeroy. Part of this is that Duke is more lightly regarded here, but the fact that all of the next six teams by seed in their region are rated at least as high here as by Pomeroy plays a role as well. West Virginia, despite having to get past Kentucky just to get to the Final Four, is considered to be more likely to win it all.
- Kentucky also sees a big jump in their chances of winning it all here, in large part due to difference of opinion on Wisconsin (who's actually the favorite, by a narrow margin, to escape the East according to Pomeroy's numbers).
- Both systems rate us about equally (24th Pomeroy, 26th Bradley-Terry). The main differences in our percentages are due to our opponents: New Mexico State is considered a much tougher opponent by this system (79th) than it is by Pomeroy's (115th), reporting a 23.8% chance of upset compared to 14.2%. But Maryland is not so highly regarded (10th Pomeroy, 22nd Bradley-Terry), so this model gives us a slightly better chance of reaching the Sweet 16. Then we meet the buzzsaw named Kansas, and Bradley-Terry considers them a much bigger busszaw, so our chances thereafter drop accordingly.
- Pomeroy's formula has Wisconsin tied for 3rd most likely to win it all (with Syracuse, behind Duke and Kansas). This one has them 21st.
- Just two 10 seeds (and no one lower) are favored in the first round by this method: St. Mary's and Missouri. Pomeroy favors Old Dominion, Utah State, Georgia Tech, and St. Mary's.
- Poor Arkansas-Pine Bluff. Pomeroy's model gives them a 1 in 2.5 billion chance of winning it all; this method isn't even that generous, at 1 in 25 billion. (That may be an artifact of their calculations simply being for the "play-in winner", which could have been Winthrop, who is rated slightly higher.)
I don't mean this to supplant Pomeroy's rankings and the analysis by the group over at Basketball Prospectus, of course; their data factors in scores and not just win-loss records. But by ignoring record entirely, their analysis also loses some information. Losing (or winning) close games isn't all luck, and a team that consistently failed in the clutch is more likely to do so again. Ideally some hybrid of the two could be used, perhaps by averaging the two rankings in some way before applying log5.
Finally, some major differences among the RPI and Bradley-Terry rankings:
- Teams the RPI overrates: San Diego State (19th vs. 38th), Cal (20th vs. 42nd), Siena (27th vs. 48th), Old Dominion (28th vs. 50th), Utah State (32nd vs. 54th), UAB (42nd vs. 58th), Wichita State (45th vs. 60th), Kent St. (47th vs. 74th), Oakland (51st vs. 80th), New Mexico State (52nd vs. 79th). Common thread, for most: Lots of games against the 100-200 range and/or low 200s, very few against the high 200s and 300s. Oakland seems to be the exception; theirs may be due to the opposite effect of playing teams way over their ranking and getting a huge SOS boost to offset the effect the loss has on RPI.
- Teams the RPI underrates: Gonzaga (37th vs. 25th), Missouri (46th vs. 33rd), Notre Dame (48th vs. 30th), Cornell (49th vs. 35th), Marquette (50th vs. 27th), Virginia Tech (58th vs. 36th), Ole Miss (62nd vs. 47th), Seton Hall (68th vs. 43rd). Again, there's a common theme: Tons of games against "RPI anchors" which aren't (for good teams) functionally that much different from playing teams in the 150-200 range but are punished far more severely in the RPI.
This seems to confirm what conventional wisdom has always said: RPI puts a big emphasis on choosing your cupcakes carefully. Choose poorly, and your RPI will suffer even if you win. Choose wisely, and you can get a bunch of near-guaranteed wins without making your strength of schedule look too embarrassing. This year, Virginia Tech chose ... poorly. And it is the reason they didn't make the tournament.
This is a FanPost, written by a member of the TOC community. It does not represent the official positions of The Only Colors, Inc.--largely because we have no official positions.
6 comments
|
0 recs |
Do you like this story?
Comments
Actually doing the rankings is pretty easy
I wrote a Python script that understands how to read Ken Pomeroy’s “games played” database. Then it iteratively updates the rankings (starting from an “estimate” of all teams = 1) based on the previous estimate of each team’s rating and their opponents’ ratings. Generating new ratings is as simple as downloading the most recent version of the database, then kicking off the script. Running it takes about 5 seconds (for 100 iterations, which should be more than enough for the ratings to converge). Generating the log5 table is a little trickier, as you have to make it aware of the potential opponents at each round, but in all cases the chances of winning in one round are equal to the chances of getting that far in the first place, times a weighted average of the chances of beating each potential opponent (weighted by the chances that that opponent will get there). I did that in Excel but could probably whip up a short addition to the Python script that would do it if you input the bracket in some simple format.
A technical explanation about the math behind the ratings can be seen here. For the procedure of calculating them, each iteration updates strength of schedule based on the previous ratings, then multiplies by (W+0.5)/(L+0.5) to get the team’s new rating. Strength of schedule is calculated as the rating of a team against whom you would have the same expected record as you have against your actual schedule (plus a tie against an average – rating 1.0 – team thrown in to avoid zeros and infinities). This, surprisingly, is a function of your own record as well; though counterintuitive, this is necessary to avoid the pitfalls of the RPI. There’s little functional difference for Kansas if they play five cupcake games against mid-level teams like Morgan State or Vermont or if they play those five games against the likes of Bryant and Alcorn State – they’re still overwhelming favorites to win all five, and they should not be punished much more severely for the latter than the former. On the other hand, Vermont facing Morgan State instead of Bryant is significant – it’s a game they could legitimately lose instead of one in which they should cruise easily.
A sample calculation: Two teams have each played teams with ratings at 2.0 and 0.5 (plus the fictional tie). One went 2-0, the other 1-1. Let us assume the estimated ratings are 5.0 and 1.0. SOS calculations: For the 1-1 team (estimated rating: 1.0):
Chance of beating the 2.0 team: 0.333
Chance of beating the 0.5 team: 0.667
Chance of beating the (fictional) 1.0 team: 0.500
Total expected record: 1.5-1.5 (win ratio: 1.0)
SOS = estimated rating / expected win ratio = 1.0
Updated rating = (1.5/1.5) * SOS = 1.0
For the 2-0 team (estimated rating: 5.0):
Chance of beating the 2.0 team: 0.714
Chance of beating the 0.5 team: 0.909
Chance of beating the (fictional) 1.0 team: 0.833
Total expected record: 2.457-0.543 (win ratio: 4.522)
SOS = estimated rating / expected win ratio = 1.1057
Updated rating = (2.5/0.5) * SOS = 5.5286
Of course, the 2.0 and 0.5 teams are going to have their ratings adjusted as well, so the next iteration may well correct back in the other direction.
This is not what you get when you look up "pretty easy" in the dictionary
Fight for The Only Colors: Green and White!
by KJ@theonlycolors on Mar 17, 2010 7:29 PM CDT up reply actions
Well, writing the script only took half an hour
That does all the number crunching; all I have to do is tell it where to find the games database.
Vegas gives MSU 2.4% odds of winning the whole thing—vs. a high of 1.2% for the formula-based odds.
Fight for The Only Colors: Green and White!
by KJ@theonlycolors on Mar 18, 2010 7:23 AM CDT reply actions
I entered these ratings in my ratings comparison bracket group
FYI, I filled out a bracket based on these ratings, as well as some others (Pomeroy, Sagarin, TeamRankins, etc), and created an ESPN bracket group to track how they perform. Details can be found here:
http://audacityofhoops.blogspot.com/2010/03/prediction-system-bracket-challenge-rd.html
The actual group is here:
http://games.espn.go.com/tcmen/en/group?groupID=123804&entryID=0
I’ll be posting updates after each round.
by TheAudacityOfHoops on Mar 20, 2010 1:47 PM CDT reply actions
Fun idea
Thanks.
Fight for The Only Colors: Green and White!
by KJ@theonlycolors on Mar 20, 2010 2:55 PM CDT up reply actions
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
