Tuesday, January 20, 2009

The Fallacy of "Knowing" How to Win

Often when a team wins or loses by five or less points, you'll hear a commentator talk about how this team performs X, Y and Z under pressure. Some virtues are also normally attached to the coach involved as well. Some coaches just know "how to win" close games, while other players and coaches wilt and crack under pressure.

Hey, why let the facts get in the way of a good story? When you actually look at win-loss records for games decided by five points either way, the picture isn't so clear, and it suggests that luck plays a great role in the outcome of games. Specifically, I'm looking at the past five years. The following group has one sure Hall of Fame coach, one borderline candidate, one coach on the hot seat, and two coaches who were fired after the 2008 season. Quick, who are these coaches?:

Coach 1: 20-15 (0.571)
Coach 2: 29-23 (0.557)
Coach 3: 25-25 (0.500)
Coach 4: 14-16 (0.467)
Coach 5: 17-22 (0.436)

The first coach is Duke's Mike Krzyzewski, which may or may not be surprising to you. It may not be because he is first in this mini-sample. It may be surprising because his overall win percentage is 0.809 in that time frame. While he does seem to be the best of this bunch, his edge isn't that great over the second coach...

... Who is Ben Braun of California, who was fired following the 2008 season after a second straight year among the bottom of the Pac-10. Despite an overall record of 79-75 (0.520) in that time period, Braun posted a 29-23 record in close games, a slight improvement.

Coach three is Jim Baron of Rhode Island, who has gotten a reputation for being an average / mediocre coach. His overall record in the last five years is 80-76. His 25-25 record in close games seems to support that theory, except that the next coach on the list is...

... Michigan State's Tom Izzo. You might know him as the man that won an NCAA championship in 2000. His Spartans have made the tournament every year from 1998 to 2008. Yet he has done this despite a record below .500 in close games. His overall winning percentage in the same time frame is 0.691.

The last coach is Providence's Tim Welsh, who was fired after several years of mediocrity with the Friars. While his 17-22 record in close games isn't great, it isn't that different from Izzo's. His overall winning percentage for the past five years is .530.

Although it is only a five-team sample, I don't doubt that running the numbers for more teams or more years will produce more inconsistencies, and a general trend toward .500. This mirrors the work done in other sports when it comes to margin of victory - Baseball teams with an unusually good record in one-run games one year will often see the total flip-flop the next year despite keeping the same personnel.

When columnists, analysts and color commentators love to harp about the heroic clutch abilities of some teams to pull out games, it obscures the real reason why Michigan State and Duke are among the elite college teams - They beat the snot out of weaker teams and stay out of tight scores. It seems odd to say, but in a close game, an elite team isn't much better than a 50/50 shot to win the game. In a five-point game, all it takes is two possessions for a weak team to sneak out a win.

Sunday, January 11, 2009

Player Improvement By Class Year

I'm going to go in an opposite direction today from the past couple posts. In those, I looked solely at a team's performance. Today, let's look solely at individuals, and specifically, how they progress and develop as they age.

The best advanced stat I've seen that encapsulates a player's overall value would be PER - player efficiency rating. It was developed a few years ago by John Hollinger, who now does a lot of work for ESPN.com evaluating pro basketball. It was created by him to properly weigh offensive statistics - shooting percentage, turnovers - that typically get lost in the shuffle by just looking at points per game. It does take into account steals and blocks, but primarily, it reflects a player's offensive contributions. For more on PER, click here. It is also kept for college games now, in contrast to when I had to compile and calculate by hand the numbers I used for a primitive site a couple years ago. Click here if you want to read some juvenile writing by me about some bad Rhody teams (and apologies about the screwy formatting).

With that all in mind, let's try to answer a seemingly simple question - How much do players improve from their freshman year to their senior year? And, if we can, can we find a pattern among players that improve?

As far as methodology goes, I looked at the class that started as freshmen for the 2004-2005 college season. To qualify for a year, a player had to play at least 10 games and at least 10 minutes per game. It's fairly low threshold, but it still produces a pretty significant quantity of players - 64 freshmen in 2004-05, 86 sophomores in 2005-06, 111 juniors in 2006-07 and 109 seniors in 2007-08. The PERs by year:

Freshmen - 16.32
Sophomore - 19.17 (14.87 percent)
Junior - 20.71 (7.44 percent)
Senior - 21.58 (4.03 percent)
Total cumulative change - 5.26 (24.37 percent)
Average NCAA player - 19.85 PER

Essentially, the data means that an AVERAGE college player will improve by 1/4 from his freshmen to senior year, as long as he's good enough to get 10 minutes per game in his freshmen year.

But the distribution of this improvement is important. While there are obviously going to be exceptions and outliers, the most improvement takes place from freshmen to sophomore year. The total improvement - almost three points of value - is more than the improvement from sophomore to senior year combined. Meaning, if you aren't a star in your freshman year, you probably aren't going to be one in your senior year either.

There have to be exceptions to this though, right? Let's take a closer look at the data, maybe at some guys who were awful as freshmen. Here are the 15 players with the lowest freshmen PER, and what they ended up with as seniors:

Parfait Bitee - Rhode Island - 7.5 - 18.6
Derrick Low - Washington State - 8.5 - 19.6
Joe Crawford - Kentucky - 8.6 - 19.5
Drew Neitzel - Michigan State - 8.8 - 21.1
Marshall Brown - Missouri - 10.2 - 13.4
Jamar Butler - Ohio State - 10.3 - 21.1
Jonathan Wallace - Georgetown - 10.3 - 19.1
Kyle Weaver - Washington State - 10.4 - 22.8
Jason Rich - Florida State - 11 - 14.8
Jeremis Smith - Georgia Tech - 11.3 - 22.2
Steven Hill - Arkansas - 11.4 - 17.7
Russell Robinson - Kansas - 11.5 - 17.1
Charron Fisher - Niagara - 11.5 - 26.9
Gavin Grant - North Carolina State - 11.8 - 20.3
Dominique Kirk - Texas A&M - 11.8 - 15.2

So, this obviously proves that players only improve by 24 percent from freshmen to senior year... Except it doesn't at all. Actually, for this group of four-year players, they seem to improve by a whopping 9.27 points, or about 46.45 percent. What's going on?

Part of the problem stems from selection bias. If you're playing regular minutes as a freshman, it normally means you show at least some promise. Also, players that are too bad to get regular minutes as a freshmen will instead become part of the study as a sophomore. Or as the numbers by academic year shows, juniors and seniors.

Let's refine our numbers a bit then, and look at just players who play significant minutes for all four years of their collegiate career. How do they fare in this study?

Freshmen PER: 16.32
Senior PER: 21.7

Hey, isn't that number eerily similar? That's right - If you look at the group of 60 players who played all four years in college, as a whole their improvement is about 25 percent. Yet the bottom 15 players, the bottom quarter, improve by a substantial margin.

One issue is that it's much easier to improve from a barely-playable freshman, or a freshman on a horrible team, to an elite level player. Of the 15 worst, none improved to an All-American level. The biggest gainer, Charron Fisher, played for Niagara, and most of his rating is tied up in his ability to create shots. He played a ton of average minutes offensively, which does have value (think Antoine Walker), but isn't something any team shoots for ideally.

So instead, let's breakdown players into ranges by 1) what they started at in PER as a freshman and 2) what they finished at as a senior.

Starting PER - Ending PER average
7.0 to 11.9 - 19.29 (15 players)
12.0 to 16.9 - 20.46 (22 players)
17.0 to 21.9 - 24.33 (18 players)
22.0 to 27.3 - 24.2 (6 players)

Once you break players up into ranges, the results become a little more clear - it's incredibly hard for a player to break through the ceiling of a 24 PER. Once you hit this level, it's harder to improve. Then again, most of the players putting up 22+ PERs as freshmen don't need to improve their games to become viable contributors. D.J. White (26.3), Chris Lofton (23.0) and Bryant Dunston (24.9) were very good from the start of their careers.

Some interesting numbers pop out when you look for analomies. The player to improve the most from freshman to senior was Georgetown center Roy Hibbert, from 15.2 to 31.1. Arizona's Jawaan McClellan (-4.9) and Houston's Lanny Smith (-4.8) regressed the most; both started in the mid-16s.

Let me try to bring it all home and summarize after looking at this group of players...

- In a proclaimation sure to make Dickie V. unhappy, if you're an elite player as a freshmen, there isn't any evidence to suggest that you're "helped" from extra years at school. Players posting PERs greater than 20, on average, didn't see significant gains by staying in school four years.

- If you're good enough to get 10 minutes per game as a freshmen, it suggests you'll be decent enough to be a solid regular three years later, even if you start out horribly. From personal observation, Parfait Bitee was about as bad as you could be as a freshman guard (PER 7.5), but he was a solid starter (18.6) by his senior year.

- ... But if your PER is too low as a freshman, it suggest superstardom isn't in your future. The lowest starting PER (15.9) that eventually ended up with a PER greater than 30 is Roy Hibbert. Most players that are eventually great start out as very good.

These are my preliminary findings, and I'll take a look at some other years in a future entry, to see if the trend holds for some other classes. Feel free to post suggestions, comments and questions.

Friday, January 9, 2009

24 Win NCAA Tournament Teams

Let's take a closer look at the 24-win group, since it's the first group that we see a significant split at. Again, the overall percentage:

24 wins - 61.29 percent (19/31)

The question at hand is whether a 24-win team with NO quality wins has made it as an at-large team. I only have three years of data to work with, but parsing through things, here are the 24-win teams that have made it to the NCAAs in that time frame:

Murray St. OVC 24-7 2006
South Alabama SB 24-7 2006
Louisville BE 24-10 2007
Oral Roberts Sum 24-9 2008
Austin Peay OVC 24-11 2008
UAB CUSA 24-7 2006
Tennessee SEC 24-11 2007
Clemson ACC 24-10 2008
Montana BSky 24-7 2006
Air Force MWC 24-7 2006
Marquette BE 24-10 2007
Cal St. Fullerton BW 24-9 2008
Old Dominion CAA 24-9 2007
MD Baltimore County AE 24-9 2008
Connecticut BE 24-9 2008
San Diego St. MWC 24-9 2006
Pacific BW 24-8 2006
Notre Dame BE 24-8 2007
Long Beach St. BW 24-8 2007

I'm doing all of this by hand, so thankfully, we can eliminate a few teams right away. Louisville, Marquette, UConn, Notre Dame, Tennessee and Clemson play in power conferences. Oral Roberts, Cal St. Fullerton, MD Baltimore County, Long Beach State, Murray State, South Alabama and Austin Peay won their conference tournament.

That leaves only a few teams. 2007 Old Dominion played in the Colonial Athletic Association, so their conference schedule is similar to the A-10, and they did receive an at-large bid. They beat Georgetown in mid-November, but outside of that their "best" out-of-conference wins are against mid-tier teams: UAB and Toledo. They got blown out by Virginia Tech, and lost to Marist and Clemson. Their bid is seemingly due to a late season, conference play run, as they won their last 11 conferences games before going 1-1 in the CAA tournament.

Montana 2006 won their conference tournament, but were seeded ahead of a few at-large bids. They had one good win, against Stanford, who did not make the tourney. The next-best team on Montana's schedule, Wisconsin-Milwaukee, was a loss, and WM did not make the tournament either.

Likewise, San Diego State won their conference tournament but earned a No. 11 seed, despite a less than stellar regular season schedule. They were blown out in December by California, Providence and Washington. Their only win against a tournament at-large team was a conference schedule split with Air Force (see below for more about them).

UAB 2006 played a fairly challenging schedule, and had some good wins - Memphis once in conference play, Oklahoma State and Old Dominion out-of-conference. Air Force 2006 lost in the opening round of their conference tournament, but had good wins against Nevada, Georgia Tech, BYU, St. Mary's, the aforementioned San Diego State and Miami.

So, yes, it's possible to make the NCAA tournament without really having a marquee, or even a decent win. But as a note, the Rams can only lose a max of three games, not counting the conference tournament, and still win 24 games. They play seven games against the cream of the A-10's crop: Xavier, Dayton, St. Joe's, UMass and Temple.

A-10 Tourney Bids

At the request of a few others, I've taken a closer look at the Atlantic 10's tournament acceptance rate (for lack of a better term). For this little study, I compiled all the records of A-10 teams from 2002 - seven years and 90 teams - and looked at which ones made the NCAA, NIT and CBI basketball tournaments. All cited records are WITHOUT postseason games factored in. Let's get to the stats...

Teams in the NCAA - 17.78 percent (16/90)
Without automatic bids - 13.95 percent (12/86)

As a reference, the four automatic bid winners were 2008 Temple (21-12), 2007 George Washington (23-8), 2006 Xavier (21-11) and 2005 George Washington (22-7). GW 2007 probably played itself on to the bubble with its tournament run, but none of the four were considered tournament locks at the time, if I recall correctly.

This alone doesn't tell us much. It's not surprising that, as a whole, the A-10 doesn't get wide acceptance in the NCAAs, especially considering the bottom-feeders involved. For example, in the seven year time span, St. Bonaventure had an average season of 9-19 (overall record 62-136). Duquesne, Fordham and La Salle have not made ANY postseason tournament in the past seven years, and the Bonnies have only managed one NIT bid in 2002.

Like the other day's exercise, let's look at NCAA tournament rate filtered by win total:

27 wins - 100 percent (2/2 - Xavier 2008, St. Joe's 2004)
26 wins - 100 percent (1/1 - GW 2006)
25 wins - 100 percent (2/2 - Xavier 2002 and 2003)
24 wins - 100 percent (3/3 - Xavier 2007, Dayton 2003 and 2004)
23 wins - 75 percent (3/4 - GW's 2007 autobid, St. Joe's 2003 and Xavier 2004)
22 wins - 100 percent (1/1 - GW's 2005 autobid)
21 wins - 50 percent (3/6 - St. Joe's 2008, autobids by 2008 Temple and 2006 Xavier)
20 wins - 25 percent (1/4 - Richmond 2004)
19 wins - 0 percent (0/5)
18 wins - 0 percent (0/9)

Mirroring the results of the NCAA teams en masse, 25 wins seems to be the magic number, and even 24 overall wins has always gotten you to the dance in the past seven years. All eight teams were clearly at-large picks. The fact that only eight teams have even gotten to the 24-win plateau in seven years shows how unique the feat is. Therefore, it's not surprising that they've all made the NCAAs.

The first level a team didn't make it was at 23 wins, and George Washington needed an eight-game win streak, with the last three in Atlantic City, to earn an automatic bid. In the same year, UMass won 23 games and didn't make the NCAAs. A poor showing in the Atlantic 10 tournament is probably to blame, as they had an opening round loss to St. Louis. The only 22 win team is an autobid.

At 20 and 21 wins, things get dicey and seemingly random. St. Joe's 2008 team had three wins in Atlantic City, but still seem a bit weak on paper, as they lost to Temple in the finals and went 3-5 in their final eight A-10 games. Their only good out of conference came against Villanova, but they played Creighton and Gonzaga tough.

Richmond's 2004 selection came during a historic year, as it featured St. Joe's run at perfection with a team led by Jameer Nelson and Delonte West. While Richmond's overall record wasn't great, they did have wins against tournament teams Xavier and Kansas, and had a tough loss in the A-10 finals against Dayton at Dayton.

Just for kicks, here are the NIT percentages for various records:

23 wins - 100 percent (1/1 - UMass 2007)
22 wins - No applicable teams
21 wins - 100 percent (3/3 - 2008 for URI, Dayton and UMass)
20 wins - 66.67 percent (2/3 - Dayton 2002 and Charlotte 2008)
19 wins - 60 percent (3/5 - Richmond 2002, URI 2004 and St. Joe's 2005)
18 wins - 55.56 percent (5/9 - St. Joe's 2002, Rhode Island 2003, George Washington 2004, and St. Joe's and Charlotte in 2006)
17 wins - 50 percent (2/4 - Bonnies 2002, Temple 2006)
16 wins - 33.33 percent (2/6 - Temple 2005 and Richmond 2008, CBI tourney)
15 wins - 50 percent (3/6 - Temple in 2002, 2003 and 2004)
14 wins and below - None (0/37)

Most of the teams that did not qualify had overall records below .500, which is a criteria for the NIT. The NIT seems to reward reputation - how else to explain Temple's three berths with 15 wins? - but is otherwise fairly random once you hit 16+ wins and have a winning record. 21 wins seems to guarantee an NIT spot though.

Wednesday, January 7, 2009

NCAA Tourney Bids and Win Threshold

The question at hand - How often does a team win 24 games and NOT make the NCAA Tournament? Or, maybe a bit more on the nose, how likely is a team to make the NCAAs based on their wins, regardless of their schedule strength?

To get at the answer, I looked at all of the 18+ win teams from the past three seasons - 2006, 2007 and 2008 - and whether they 1) made the NCAA Tournament and 2) if they did, whether it was via an at-large bid or because they won their conference tournament. The total sample size included 390 teams with 18+ wins. Team numbers by win total - 19+, 344; 20+, 300; 21+, 254; 22+, 210; 23+, 167; 24+, 130; 25+, 99; 26+, 71; 27+, 54. The number of available NCAA spots for the sample is 192.

Okay, enough of the methodology - On to the percentages:

27 wins or more - 100 percent (54/54)
26 wins - 64.71 percent (11/17)
25 wins - 75 percent (21/28)
24 wins - 61.29 percent (19/31)
23 wins - 62.16 percent (23/37)
22 wins - 53.49 percent (20/43)
21 wins - 38.64 percent (17/44)
20 wins - 23.91 percent (11/46)
19 wins - 13.64 percent (6/44)
18 wins - 13.04 percent (6/46)

Looking at the raw numbers, the cut-off point seems to be 22 wins. If you have more than 22 wins, then you have at least even odds of making the tournament. If you manage 25 wins, then you're practically a lock for dancing.

However, there are two issues complicating the percentages. One, the records are from the END of the season. For example, Air Force finished 26-9 in 2007, but this is mostly because they went 3-1 in a postseason tournament. Prior to that, they lost their last four games. Similarly, in 2008 Massachusetts finished at 25-11, but this was only because of four NIT wins. Neither team made the NCAAs, but their inflated win totals bring down the percentages. Let's re-run the calculations with their records BEFORE factoring in non-NCAA postseason tournament wins.

The other complicating factor for the percentages are some of the automatic bid winners from really weak conferences. Maryland Baltimore County and California State Fullerton both earned automatic bids with 24-9 records, but neither would have made it as an at-large team. Again, with the goal of a more accurate prediction, I'm removing them.

With those two changes, the new percentages:

27 wins or more - 100 percent (54/54)
26 wins - 71.43 percent (10/14)
25 wins - 86.96 percent (20/23)
24 wins - 65.22 percent (15/23)
23 wins - 55.88 percent (19/34)
22 wins - 47.62 percent (20/42)
21 wins - 23.81 percent (10/42)
20 wins - 15.91 percent (7/44)
19 wins - 11.36 percent (5/44)
18 wins - 11.11 percent (5/45)

Ah, that makes the picture a bit more clear. The percentage for 26 wins looks a bit off, but all four qualifiers that didn't make it - Robert Morris, IUPUI, Akron and Stephen F. Austin - played in incredibly weak conferences and lost in their conference tournaments. The same thing applies to the three stragglers at 25 wins - Marist, Appalachian State and Vermont. If you play in a mid-major or above conference, and win 25 games, the at-large bid rate is 100 percent the past three years.

At the 24 win plateau, you start to run into more mid-major teams - Hofstra, New Mexico, Virginia Commonwealth, Utah State and Old Dominion (2006) failed to qualify. However, being in a mid-major doesn't disqualify you from earning a bid at this win level. South Alabama, UAB, Pacific and Old Dominion (2007) qualified. And power conferences teams aren't immune from being left out, as 24-win Syracuse found out in 2007.

As the numbers suggest, it gets to be more of a crapshoot once you only win 23 games. Teams ranging from Big 12 Kansas State to A-10 Massachusetts to CAA Drexel get left out at this level. The first ACC team - Florida State - appears at 22 wins.

However, if you are a mid-major and want an at-large bid, you better win at least 22 games. Creighton and Southern Illinois managed to get in with 22 wins, but once you hit 21 wins in a mid-major conference, no teams except automatic bid earners are in. The Atlantic 10 isn't quite at this level yet - Xavier and St. Joseph's both got in with 21 wins, but that's the low end of the spectrum.

Once you hit 18 to 20 wins, the teams making the tournament are solely those in power conferences - Big 12, Pac 10, Big East and ACC figure in prominently - and automatic bids. The 18-game at-large teams read like a who's who of college basketball: Oregon, Alabama, Stanford, Kentucky and Seton Hall.

For those cheating and skipping to the end, the summary - 25 wins guarantees you a spot in the NCAA tournament, and 24 wins gives you at least a 65 percent chance, and higher once you subtract automatic bids from weak conferences. Anything lower than 24, and the conference you play in becomes substantially more important.