Draft PredictionMachine Learning
Predicting which college basketball players get drafted, in a Kaggle competition where fewer than 1 in 100 are picked.
Out of 14,774 player seasons, fewer than 1 percent get drafted, so the real job is ranking every player by how likely they are to be picked and finding the needles. My final model ranked them near-perfectly (0.9990 on the Kaggle public leaderboard), and the reusable pieces became my own pip package.
The problem
The Kaggle set held 14,774 college-player seasons across 62 columns, and only about 0.8 percent were drafted. At that imbalance accuracy is a trap (predicting nobody gets drafted is 99.2 percent accurate), so the metric is AUROC: how well the model ranks drafted players above undrafted ones across every possible cutoff. The data itself needed forensics first: 2,462 exact duplicate rows dropped, while players who legitimately appear across several seasons were kept after checking they were repeat careers, not copies. And heights arrived stored as dates, so "6-Jun" had to be read back into 6 feet 6 before the column meant anything.
Missingness that means something
The scouting-rank column is 67 percent empty, yet a quick LightGBM screen put it at the top of the importance list by a factor of four. My starting rule said drop heavily missing features, but exploring first overturned the rule: players with a scouting rank get drafted 2.36 percent of the time, players without one 0.012 percent, a gap of roughly 195 times. Being unranked is not a data gap, it is information about the player. So instead of dropping or averaging the holes away, every such feature kept a missing-flag and a sentinel value, letting the model read absence as the signal it actually is.
Three models, one honest pick
Experiment one raced logistic regression, LightGBM and XGBoost, each imbalance-corrected with class weights. LightGBM posted a spectacular test score (0.9966) over a shaky validation score (0.9518), the classic signature of instability, while logistic regression held steady on both (0.9905 and 0.9968) and won on trustworthiness rather than a single number. Feature choices were stress-tested the same way: three feature sets were compared (keep everything, prune by missingness rules, prune correlated pairs above 0.9), and keep-everything won because the pruned candidates turned out to carry that informative missingness.
Stacking against blending
Ensembling was my slice of the group work, and the two designs turned out to serve two different users. The stacked model is the scout who refuses to miss anyone: on test it found every drafted player (recall 1.00, zero false negatives) at the price of 31 false alarms. The weighted blend is the triage tool: it cut false positives from 43 to 26 on validation while giving up a couple of finds. Tuning both and sweeping blend weights barely moved AUROC, which is its own lesson: the design choice mattered more than the hyperparameters. Internal splits favoured stacking; Kaggle's hidden leaderboard, the closest thing to truly unseen players, favoured the 20/80 blend at 0.9990, and that is the model the report recommends.
Packaged, not just notebooked
The reusable utilities became my own pip package, my_krml_14658203, built from a cookiecutter template with source layout, tests and docs and pushed to TestPyPI, and the same package later powered the feature engineering in the Bitcoin project. The work ships as a full CRISP-DM report, four experiment notebooks and saved model artefacts, not a single throwaway notebook.
weights = [0.1, 0.2, 0.3, 0.5, 0.7, 0.8, 0.9]
records = []
for w in weights:
blend_val = w * pred_lr_val + (1 - w) * pred_xgb_val
blend_test = w * pred_lr_test + (1 - w) * pred_xgb_test
records.append({
"Blend (LogReg/XGB)": f"{round(w*100)}/{round((1-w)*100)}",
"AUROC_val": roc_auc_score(y_val, blend_val),
"AUROC_test": roc_auc_score(y_test, blend_test),
})
blend_df = pd.DataFrame(records).sort_values("AUROC_val", ascending=False)