Data Science and NCAA Bracketology – Part 3

Tuesday, April 8, 2014 - 08:15

What an exciting tournament!   If you’re an NCAA hoops junkie like me, you enjoyed the past 3 weeks of action.   From the upsets (Mercer over Duke…seriously, Mercer?) to Cinderella stories (Go Dayton Flyers!) to buzzer beaters and clutch shots (Kentucky’s Aaron Harrison against Louisville, Michigan and Wisconsin.  That’s simply insane!) ,  there is no better time of year to gather with family and friends and immerse yourself in the annual basketball festival that is the NCAA tournament.

You have to give second year coach Kevin Ollie and his UConn Huskies props for believing in themselves when just about everyone else didn’t.  A third place finish in the new American Athletic Conference along with a late season blowout loss at Louisville had fans (and the NCAA committee) doubting whether this team could make a run.   But, behind the exceptional play of guards Shabazz Napier and Ryan Boatright, they took their #7 seed and marched to a championship -- taking down three of the most storied programs in NCAA basketball.  They vanquished Tom Izzo’s peaking Michigan State team, the overall tournament #1 seed Florida and, perhaps (Michigan fans please weigh in!), the most talented freshman class in NCAA history at Kentucky.   Seven seed or not, they earned this championship.

As promised in the first and second blogs of this series, I will now evaluate the bracket-picking performance of  established industry ratings systems from  RPI, BPI, Ken Pomeroy and Jeff Sagarin  along with four custom approaches I created using Dr. Wesley N. Colley’s linear algebra method and a novel application of Google’s PageRank  algorithm.   I described and compared these ratings systems in Part 2 including a summary of their predictions with some fun bracketology discussion.   Let’s dig in and see how they performed.

All Hail Page Rank!

The Weighted Page Rank approach was the clear predictive winner.   Unfortunately, it did not win a billion dollars or even pick the national champion, but neither did any of the other models.  All of the models were surprised by Dayton’s Elite Eight showing, each presuming an Ohio St victory in the first round.   None of the models picked first round upset winners Harvard, Mercer, North Dakota St, Stephen F. Austin or Stanford.    The models were all wet on Kentucky.   7 of 8 predicted Kentucky losing to Wichita State in the second round.  The Generic Page Rank model had Kentucky losing in the first round to Kansas State!   Most of the models were similarly baffled by champion UConn.  Only the Page Rank models had UConn progressing to the Final Four – all others predicting that Villanova would send the Huskies packing in round 2.

The following table breaks down the performance by model by round.   The points awarded are based on the ESPN point system with first round winners awarded 10, then doubling for each consecutive round resulting in 320 points for picking the champion.

 

Points for Correct Picks

Model

Round of 32

Sweet 16

Elite 8

Final 4

Semi-finalists

Champion

Total Points

PRWeight

230

220

200

160

160

0

970

PRGeneric

220

200

200

160

0

0

780

Sagarin

240

200

120

80

0

0

640

BPI

230

200

120

80

0

0

630

LAGeneric

240

180

120

80

0

0

620

LAWeight

240

180

120

80

0

0

620

RPI

230

180

120

80

0

0

610

KenPom

260

180

80

80

0

0

600

Ken Pomeroy’s system predicted 26 of 32 games in round 1 to take an early lead, but quickly faded --picking only 9 Sweet Sixteen and 2 Elite Eight winners and finishing last overall.   RPI, BPI and Sagarin did not fare much better.  Of the 4 industry ranking systems, Sagarin won, but not by much.   All of the industry models picked Florida in the Final Four but none had UConn, Wisconsin or Kentucky.   It is no surprise that the industry models performed nearly identically.  Recall from our previous analysis that these systems were inter-correlated at a near statistical equality of .99.    

The Linear Algebra approach did not differentiate from the four industry ranking systems finishing dead center amongst them.     In fact, the LA models (weighted and generic) scored identically in each round.   Perhaps my game weighting approach needs tweaking.

The Page Rank models started poorly, finishing last and second to last after round 1.   But, as experienced bracketologists know, bracket pools are not won in round 1 when so many points are at stake in the Elite Eight and beyond.   The Weighted Page Rank model had the most correct picks for every round after the first.  It’s not surprising that Page Rank’s results would diverge from the industry systems since it was the least correlated of the group.   Page Rank zigged when the others zagged.

The Page Rank models did not pick “consensus” Florida for the Final Four, but did pick UConn and Wisconsin.     In fact, Weighted Page Rank predicted a UConn-Wisconsin final gaining 160 points for the UConn semi-final victory over Florida.  It came within 6 seconds and 1 point  (I feel your pain Badger fans…) of correctly picking both finalists and having a chance of picking Wisconsin as the national champion!

Dissecting Weighted Page Rank’s Picks

Figure 1 shows the Weighted Page Rank bracket selections.  (Special thanks to CBSSports.com)  

Predicting a UConn-Wisconsin final was key to Weighted Page Rank’s victory.  Why were UConn and Wisconsin favored?  Recall that the Page Rank algorithm rewards victories against quality opponents much more than it penalizes for bad losses.  In fact, losses are only detrimental to a team due to the fact that they are not a win.  So, UConn’s blowout loss to Louisville late in the season was seen as a missed opportunity by the PR algorithm, not as a reason to lower UConn’s rating strength.   Likewise, Wisconsin’s late season loss to a middling Nebraska squad and their semi-final loss in the Big 10 tournament to Michigan State did not hurt so much as it didn’t help.

If quality wins are so important, then what caused Wisconsin and UConn to be highly rated?   The Wisconsin answer is easy.  They beat both ACC champion Virginia and Big 10 champion Michigan on the road.  Road wins factor 2.33 times more heavily than home wins in the weighting model.     UConn’s case is more challenging as their quality wins are primarily in-conference -- taking  5 out of 6 from AAC leaders Memphis and Cincinnati.  So what gives?  Is there a better explanation?

Florida, Florida, Florida…

In a déjà vu moment, I found myself repeating the mantra from election night 2000 when the Bush-Gore decision came down to one state:  Florida.   The primary reason why Wisconsin and UConn were so highly rated was because they were the only teams to beat Florida in the regular season.   

Florida was the NCAA selection committee’s overall top seed and was ranked first or second by every rating system.   They were clearly one of, if not, the strongest team in the country.   The Page Rank algorithm is based on building a directed graph of the season and then letting its recursive formula converge by increasing the strength of referenced nodes relative to the strength of their referencing nodes.   In the case of Wisconsin and, especially, UConn, each time Florida’s rating increased so did theirs.  They were effectively turned into rating parasites – gaining power from the increasing strength of their host!

To determine how impactful their victories over Florida were, I removed the games from the season log and reran the Weighted Page Rank model.   The results were stunning.   Wisconsin still makes it to the Elite Eight but loses its Final Four bid to Arizona.   UConn drops off the radar with a predicted loss to Villanova in the second round.   The new Final Four becomes Arizona, Kansas, Wichita State and Virginia with Arizona defeating Kansas in the championship.   Perhaps the stability of the Weighted Page Rank model must be questioned if removing only 2 games can cause such a change in predictions?  Or, perhaps not.   Those games clearly were very important this season.  Whichever way you interpret this, the fact remains that only the Page Rank model predicted UConn’s success.

Comments on Bracketology

In my previous blog I dissected how the ratings systems would have answered some common bracketology questions.  Let’s see how they fared.  

Were the best 36 at-large teams selected?

The ratings systems concurred that SMU should have been selected and that NC State should have been bounced.    SMU demonstrated its worthiness by making it to the NIT championship only to lose by 2 points to a feisty Minnesota squad.  NC State won their play-in game against Xavier signaling that there was at least one team that was less worthy to be in the field.  However, NC State went on to snatch defeat from the jaws of victory in its round 1 matchup with St. Louis.    Despite a 14 point lead, they couldn’t buy a free throw in the final minute and lost to SLU in overtime.    On the night, NC State missed 17 charity tosses …yikes!  

On the flip side, the Page Rank models suggested that Tennessee should not have been in the field.  Tennessee not only won their play-in game but made it to the Sweet Sixteen before succumbing to a strong Michigan squad.   Perhaps Page Rank is better at picking brackets than it is at picking tournament fields!

How well did the selection committee seed teams?

The consensus of our ratings systems suggested:

  • Michigan State and Louisville merited better seeds than 4

  • Virginia should have been a 2 not a 1

  • Duke should have been a 2 not a 3

  • Syracuse and San Diego State were overseeded as a 3 and 4 respectively.

Well, we got a few right.   Michigan State made it to the Elite Eight indicating that they were worthy of a 2.   Virginia’s loss to Michigan State in the Sweet Sixteen further corroborates the story.    Louisville lost to Kentucky in a Sweet Sixteen game that went to the wire, suggesting, perhaps, they may have been worthy of a 3 seed.   Syracuse’s loss to 11 seed Dayton in round 2 validated our overseed critique.   San Diego State’s loss in the Sweet Sixteen to 1 seed Arizona suggested that they were worthy of a 4.  Duke’s first round loss to 14 seed Mercer was completely unexpected.  Duke was not underseeded as a 3, they were grossly overseeded!

Were the best conferences properly rewarded?

Pre-tournament consensus viewed the Big 12 as the best conference in the country.   It received the most bids with 7.   Our analysis concurred with the experts as we fully supported the Big 12 selections.  Alas, not one Big 12 team made it to the Elite Eight!    The experts (as do I) suggest that results would have been different if Kansas and Iowa State had not each lost a key starter to injury.   Kansas had a track record to contend for the National Championship and Iowa State was peaking after winning the Big 12 tournament.  (Note to self, figure out how to include key injuries in next year’s ranking system.)

The Atlantic 10 also received accolades and 6 bids.   Our analysis concurred.  Unfortunately, the conference showing was mixed.  Dayton made a surprise run to the Elite Eight, but of the other 5 entrants only SLU could muster a round 1 victory (barely!) with highly-seeded, conference leaders VCU and UMass both upset.

A Profitable Experiment

I hope you have enjoyed reading this series of blogs as much as I have enjoyed writing them.   My intent was to demonstrate an application of data science techniques and technologies using the fun domain of bracketology.  We learned how Pentaho’s Data Integration technology could be used to prepare data for analytics.  We demonstrated the use of R for matrix manipulation, graph analytics and statistical plotting and visualization.   By integrating these free/low cost open source technologies, a data scientist can create a workbench for data wrangling and statistical analysis.   We added in some domain expertise and conducted an experiment in the real-world laboratory of NCAA basketball to make this a complete case study.

I truly did not expect the Weighted Page Rank algorithm to be the predictive winner.  That said, I used it in a friendly bracket pool and…won!    I am not sure if it was a fluke, but I am a few Benjamin’s richer.   Perhaps this season’s results, seeding, and upsets produced the perfect storm for the Page Rank approach?   Or, maybe not.   I guess I just figured out my next data science project:  assessing Page Rank’s performance for seasons past.  

 

Contact us today to find out how Inquidia can show you how to collect, integrate and enrich your data. We do data. You can, too.

Would you like to know more?

Sign up for our fascinating (albeit infrequent) emails. Get the latest news, tips, tricks and other cool info from Inquidia.