Arturo avatar

The 2014 Draft Preview Part 1: The NCAA Numbers

 

"The tactical result of an engagement forms the base for new strategic decisions because victory or defeat in a battle changes the situation to such a degree that no human acumen is able to see beyond the first battle. In this sense one should understand Napoleon's saying: "I have never had a plan of operations."
Therefore no plan of operations extends with any certainty beyond the first contact with the main hostile force."

-Helmuth von Moltke the Elder

Every year at around this time, I break out the tools and proceed to analyze and shed light on what is one of my passions. I am always up for writing about the NBA draft.

Welcome then, to the start of my fifth annual draft preview and rankings, where I take it upon myself to write, project, and speculate about the NBA draft using a surprisingly effective draft model to predict player performance using data publicly available on the internet.

We've had some successful graduates. Using this data, I built two models to predict the future performance of NBA draft picks (go here for the model build parts 1 & part 2 ). In very general terms, the models use the available data to predict future performance for each player coming into the draft from the NCAA. Based on that prediction, a ranking is done and a draft recommendation is generated.

It has performed at a very high level. For the full history you can go to:

A neat little summary of how the model does is this:

As always, we are going to try to make it bigger, better and 100% more awesome.

I think this year we outdid ourselves. We start with the NCAA numbers. We will follow with the international numbers in part 2 and finally, the full model with draft recommendations in part 3.

Before we get to the numbers, I need to adress the elephant in the room. You see, I had a plan and it was going to be great. I had all my draft numbers crunched and I was quite ready to wax lyrical about one of the truest truisms in basketball. You see, building a basketball team can be simplified down to the art of properly managing the short supply of skilled tall people. This draft then offered a stark and obvious choice to a team that has meandered away their draft capital. 

The Cavs are on their third number pick in four years and have very little to show for the first two so far. The third then should obviously be spent wisely on the surest possible thing in the NBA, the franchise big. We've seen this story again and again in the NBA. Russell, Wilt, Kareem, Shaq, KG, Duncan, the road to the title is built on these dominant big men and the Cavs looked to have found their man in Joel Embiid.

Then this happened:

If you don't follow Jeff Stotts (@RotowireATC) and his fabulous instreetclothes.com injury blog, shame on you. The long and the short of it is Embiid suffured a foot injury that will require surgery and takes him out for the year. The injury is one that may indicate a broader systemic issue that could shorten his career and severely limit his production. Or it might not. Were I advising Cleveland, knowing the particular adversarial theological relationships the city's franchises hold, I would have to advise them to avoid this kid like the plague without thorough medical assurances that this is not going to be an ongoing problem.

Bummer. There goes my initial narrative and a potential franchise cornerstone for the Cavaliers.

Let's get to the numbers to see why. This year I'm doing things slightly different so I'll make with the explanations first. With a large assist from Professor Dave Berri over at Wages of Wins, I put together the Wins Produced, Win Score, Position Adjusted Win Score for every player in the NCAA for the 2013-14 season.

  • Wins Produced/Wins (http://wagesofwins.com/how-to-calculate-wins-produced/) is of course our main regression based metric on this site. Player value will be shown here shown as the W/L% contribution for a player if he played 48 minutes. In simple terms, a team composed of .100 (10%) players would be expected to win half their games. A team composed of .200 (20%) players would be expected to win all their games (obviously, there are diminishing returns). I've also taken the time to adjust for strength of schedule for each player as well (a big issue in the NCAA).
  • Win Score/Position Adjusted Win Score: Simplified Version of Wins Produced. It’s a measure of net positive things happening on court as a function of a player’s boxscore statistics.  Win Score =  Points + Steal + Offensive Rebounds + 0.5*(Defensive Rebounds +Assists + Blocks) – Turnovers – Field Goal Attempts  – 0.5*(Free Throw Attemps + Personal Fouls). For Position Adjusted Win Score (PAWS) we simply subtract the average performance for players at each position.
One thing that's new is the idea of Ratings.  For each measure we look at, I’ve calculated the mean and standard deviation for all players with a minimum of 12 minutes played per each game played by their team. I’ve then used that to work out the percentile value for each player stat. In layman’s terms, a player with a 10 Rating is better than 10% of all players at his position in the particular stat or component. A 50 Rating means an average player. A 90 Rating means better than 90% of all players. 
 
You will then see a Wins Rating, a Wins Score/PAWS40 Rating and a Net Rating which combines the two. You will also see ratings for every relevant player statistic (spoilers). 
 
The NCAA Wins Produced numbers then look like so: (Editor's note: These are just the NCAA numbes and not the final draft projection. Those are coming later)
 
That table includes all ranked prospects from Draft Express as well as the top 50 unranked seniors.
 
A sorted version of that table looks like so:
Lot's of good information on all the players but it doesn't really provide a lot of detail. What if I gave you the tools to find that detail?

NCAA Interactive Player Ratings

 

That table has every single qualifying NCAA player and shows their rating for every single important category so you can see why the overall numbers like or dislike them.

But what if you wanted to compare players? Knock yourselves out:

NCAA Player Comparison Tool

That should be more than enough to get your feet wet. Next up, international prospects,but you'll have to wait for that until Part 2

-Arturo

I spent yesterday afternoon looking at the 2012 draft projections you did. I wanted to see how the teams and players fared two years out.

It's hard to say for a lot of the players you ranked highest. Guys like Mosley, Sanders, Horton, and Gordon never got drafted. How would these guys have fared if they were picked where you ranked them? Hard to say, but the fact that none of them are in the league isn't encouraging.

NBA GMs almost NEVER drafted players you ranked highly in the first round. So, how'd they do?

Well, 26 of the 30 picks went to players you guys had ranked lower than where they were chosen. Of those 26, 11 registered WP48 of .099 or better. So 11 out of 25 times, NBA GMs came away with a player that was average or better, two years after the draft.

Three of the remaining four first round picks went where you ranked them. Two of those three -- Davis and Lillard -- were above average last year. So, in all, NBA GMs came away with above average picks a little less than half the time.

The one guy who was chosen LOWER than where you ranked him -- Jared Sullinger -- was well below average last year: .028 WP48. He fell because of back problems. Then had a pretty good rookie year -- before he missed half the season with, you guessed it, back problems. He hasn't been the same player since. Brad Stevens played him out of position quite a bit last year, but that's not his problem. The fact that he took nearly 25% of his shots from three point range where he shoots a low percentage may have had something to do with it. Is that because he doesn't want to muck it up inside and hurt his back again? Maybe.

Your model fared pretty well in the second round. Seven times NBA teams drafted players you ranked higher than where they were picked. Five times they were rewarded with picks that registered WP48 of .089 or better -- so , average to above average. Draymond Green at .191 was probably the biggest steal, but Kyle O'Quinn at .136 wasn't bad either, although you guys ranked him 45 and predicted a pretty low first year WP48.

Your model had some big misses. Andre Drummond was the biggest. You guys gave the Pistons a D for that choice. No one could have predicted that he'd post a .331 wp48 -- highest of anyone not named Chris Paul -- but you guys had him at 67! OOPS! Brad Beal and Will Barton haven't exactly burnt it up, either.

Here's the bottom line: Last year, 18 of the guys who were drafted in 2012 -- both rounds -- registered WP48 of .099 or better. Of those 18, only four were ranked more highly by your model than where they were drafted. Three were drafted where you ranked them. 11 were drafted higher than your ranking. By those criteria, it looks like NBA GMs did better with whatever model they were employing.

What say you?


As a stats-minded guy but not an expert, and a bball fan but not an expert, what should I be taking away from this? That you're stating the top 5 picks should emphatically be some combo of Khem Birch, Joel Embiid, Kyle Anderson, Javon McCrea, and Jarnell Stokes?
Will Barton has shown good improvement last year and at the 40th pick, has been considered a good draft choice. Not a Chandler Parsons good pick, but a good pick regardless.

Does the analysis take college strength of schedule into account when evaluating player stats? I could see this being a big factor on how pure athletes rank.
One of the things that can be a game-changer between a projection and actual performance is position (and more specifically, the role that comes with the position).

Sticking with guys I know well as a Michigan fan: Here, Stauskas is evaluated as a 2.4. That's fair. In Michigan's four out one in, Nik played as an interchangeable 2/3. In the pros, however, he's probably going to play SG almost exclusively, and possibly a little PG.

Delving into the cool tools here, you can see Nik's main weakness is rebounding, and if he's playing a lot of minutes at SF, that might be concerning. But Nik almost certainly won't be playing that position in the pros, so it won't be as pronounced as it appeared in college, and as a pro, he'll be evaluated against other guards, not forwards.

(There's also an entire conversation here about U of M de-emphasizing rebounding in its scheme, coupled with guys playing out of position. And another conversation about the fact that Nik measured better than you'd expect given his rebounding and steal numbers, which I think supports the argument that he's more athletic and capable of more than what he was asked to do at U of M... but, I digress)

Here, Nik ends up rated 107th...which to me, is almost unbelievable. He's one of the best shooters in the game, as evidenced by the 97 rating on True Shooting. I don't see a single reason why he can't play the kind of role in the NBA that guys like Reddick or Korver play - particularly if he drops later into the first round to a team that will allow him to develop over time rather than rushing him into a scoring role.

To me, a guy with those kinds of tools is certainly more valuable than 107th.
pmassari,
You have to evaluate the model versus the first four years of a player's nba career. I'll point you to the latest evaluation I did last year:
http://wagesofwins.com/2013/06/27/2013-nba-draft-extravaganza-rev-3-eliminating-the-big-man-bias-the-euro-numbers-and-the-cheat-sheet/

Functionally, the draft is a 50/50 proposition and the model makes it 80%+ historically. Yes there will be misses but better odds is an edge. There is also strong evidence that there are many nba caliber talents left undrafted.

BigPhil,
This is just the initial numbers. Full recommendations come in Part 3. However, if a player is exceptional in the NCAAs, it's generally a good indicator of future performance.

James,
The Wins Produced number is schedule adjusted.
Van Treese is one of my favorites in this draft. Played on a loaded louisville team all four years. Great at using/generating possessions
(shooting efficiency, offensive rebounds, and decent steal rate)
Another thing that gets left year after year is the player's previous years. Some players might have started off more efficient but then leveled off or reduced in production. A couple of guys' draft blogs take that into effect. In model this matters given the effects age has.
Andrew,
This is just part 1. Bigger and Better remember?
I love this stuff since I want to be a scout or part of an analytics team. I don't see one of my favs for next season, Jordan Parks, in the database.
Great stuff as usual @ArturoGalletti!
Andrew,
Change the Draft Group to All. Jordan Parks has an awesome rating.
Compared to conventional wisdom, you seem to be strongly over weighting seniors and juniors. Do you correct at all for:
- age?
- number of years of college ball played?
- strength of schedule?

If you could add those factors, I wonder if your results would improve?

Some of the players who you rank highly have improved dramatically over their four years at college (e.g., your #1 Ryan Watkins seems to have made huge strides in rebounding from his rookie to his senior year). If you left, say, Andrew Wiggins in college for four years, would his numbers improve comparably, and would he then be better than Ryan Watkins in your ranking?
KuH,
This is just the basic numbers. The Final model will account for all of that.
Arturo, I looked at the link you provided. There's a lot of data there, and my question is pretty simple: Did you guys do a better job of identifying average or better players or did the GMs?
Mitch McGary seems to be missing from the list. He is ranked 28 by DraftExpress, omitted with the international players would be my guess.
pmassari,
See the table I added to the post. typical draft hit rate is 50/50 for all picks, 70% for Top 3 pick, 68% for Top 5, 61% for top 10. Model hits at 85% to 90% on all players drafted outside the top 5 liked by both models.

The answer is a clear yes.
InkedGeologist,
Internationals are coming on Monday.
Nice work. I continue to find it interesting that Jabari is getting rated like he is both in contention for college's most dominant player and most skilled player, when in reality, neither of these things are close to being true. (I know it's outside of numbers based work, but the Jabari is freakishly skilled thing kills me. How is everyone's brain ignoring that Jabari was probably worse at shooting, passing and dribbling the ball than Wiggins this year, who is called a raw, desperately needs to improve skill level athlete - and who is overrated in his own right!)
But McGary went to Michigan and was born in Indiana.
Inked,
Check now.
Super curious about Giannis. We never got real international evaluations last year. Pretty sure he got drafted on youtube clips alone.
Hi Arturo. Sorry, what does "70 percent for a top three pick," mean? That the model's picks register a four year wp48 of .99 or better? Same for 85-90 percent outside top 5? It helps if you can unpack some of the stats language.
Arturo - thanks for the explanation. It wasn't clear to me - and probably other readers - that this post was only the 'raw data', and that your real ranking would come in Part 3. You may want to change the title to avoid confusion.
Pmassari,
I'll put an explanation in part 3. Too long for a comment section. The short answer is that the stat based model shows a significant edge over time to the market.
Thanks, Arturo.

Listen, I'm a Celtics fan too. I'm also a guy with a finance degree, so I'm down with what you're doing here.

I totally get the 4 year thing. Still, you do give a projected WP48 on your draft cheat sheets. I crunched the numbers for 2013 and can send you the spreadsheet if you like.

Last year was lousy for rookies. Most didn't get enough time for their wp48 to make any sense. Still, the most productive rookies in the NBA last year -- Plumlee and Dieng -- were rated flat BUSTS by the WOW model. You also made a point of saying you didn't want either of these guys on the Celts!

In fact, most of the most productive players -- such as they were -- were rated busts by the WOW model, while the guys rated great or good generally underperformed.

To be fair, Noel and Porter missed most of the season, although they were taken pretty high anyway, so there's really no conflict between the model and GM behavior. Also, who knows how the great or good tier guys would have done with more playing time?

Again, I believe that quantitative models should inform GM's decisions in the draft -- and a lot more than they do now. I need more convincing, though, that teams would be better off navigating by this model than by what they're doing now.
@pmassari

I don't understand why you're getting hung up on the projected WP48. You say that you're comparing Arturo to GMs, but you're judging Arturo's model on criteria that you can't compare to GMs since they don't give performance projections on prospects at all. The thing that everyone should understand, if one reads Berri's draft coverage at all (college WP48 has an r^2 in the 70s when regressed against NBA performance IIRC), that college performance does not cleanly translate into NBA performance. So any possible projection of college prospect performance in the NBA is very likely to be noisy. Luckily, absolutely nothing hinges on anyone's ability to predict a player's WP48 from their college numbers (unless they happen to be making some oddly specific non-traditional wagers), so it doesn't really matter. What matters is that the model is able to rank players relative to each other better than whatever models the majority of NBA teams happen to be using. On this count, perhaps Arturo could be more explicit, and perhaps even exposing the projected WP48 when it's not even meant to be strongly predictive is confusing, but these are at best mild criticisms of Arturo's explanations, and not of his modelling capabilities.

Regarding your first post... Well, Arturo perennially ranks a lot of NBA prospects using an objective model. This is informative, but it's problematic to compare that directly to the draft selections of NBA teams. Again, NBA teams don't unveil their objective models, so you don't get to see where all of the never-going-to-be-drafted-in-a-million-years prospects show up in their rankings. Consider the above rankings since it's handy (not that they are likely very reflective of Arturo's final rankings). Applying minimal interpretation of Arturo's statements in the post, a minimal amount of game theory, and for simplicity taking the listed rankings as final (even though we know that they aren't), I would expect Arturo, if he had the Cav's #1 pick, to trade down to around 8-14 so that he could have a high probability of being able to select Kyle Anderson (his 2nd rated lottery player) without paying him much above his market rate. The behavior that we observe of NBA team selections reflects this sort of flexibility. Arturo, of course wouldn't select his #1 overall rated prospect, Ryan Watkins, with the #1 pick, because he could easily pick him up out of free agency for no money, and no guaranteed contract. So, I think that your comparison methodology is off. In short, you're not comparing apples to apples.

Regarding your statement that you "need more convincing" "that teams would be better off navigating by this model than by what they're doing now", well... it's good to be skeptical, of course. But you seem to be trying to squeeze Arturo for an explanation tailored to your objections, even though they aren't clearly stated. He gave you a link to a past analysis, and you dismissed it out of hand, and then essentially asked for something better without ever specifying specific criticisms. Arturo's pretty good at responding to criticism when they are brought to him in a constructive way. He's even been known to do entire series of blog posts in response to good criticism of his work. If you do in fact want a thorough response to your objectives, I suggest that you lay out your criticisms more coherently, and in a more structured manner. To get a good response to criticism requires a little bit more work than a stream of consciousness a comment. If you're not actually trying to get a good response, then you seem to be a bit of a troll, which is unappreciated.

Sign in to write a comment.