AI in Medicine — Summer Course 2026

Conventional
Machine Learning

Theory, evaluation, and interpretation of ML models for biomedical data —
from logistic regression to random forests in ~2.5 hours

28 July 2026 9:00–12:15
🧑‍💻
Your Instructor

Koay Hong Vin

Education
B.Eng. Electrical Engineering
PhD in Artificial Intelligence — Autonomous Vehicles
Current
Risk Modelling and Data Scientist
Ant International
Links
linkedin.com/in/koayhv
hongvin.me
00 / Agenda

What We'll Cover Today

01
Foundations & the ML Landscape
~20 min · what ML is, daily-life examples, how a model learns, features
02
Classification & Logistic Regression
~20 min · probabilities, thresholds, intuition
03
Evaluation Deep-Dive
~25 min · confusion matrix, accuracy trap, ROC, metrics
04
Overfitting & Model Selection
~25 min · bias–variance, validation, hyperparameters
05
Beyond Logistic: Trees, Forests & the Toolbox
~20 min · trees, ensembles, k-NN, SVM, unsupervised
06
Clinical Safety, Ethics & Hands-on
~40 min · failures, bias, calibration & lab
01 / Concept

What is Machine Learning?

  • Algorithms that learn from data — instead of being explicitly programmed with rules. You don’t teach your phone’s face unlock by describing every face; you show it examples.
  • Classification — predict a category. Same task your email does sorting inbox vs spam, or a dermatologist calling a lesion benign vs malignant.
  • Regression — predict a number. Same as the weather app predicting 34°C tomorrow, only here we predict a disease progression score.
  • Supervised learning — labelled examples (input → answer). Both classification and regression are supervised. Like learning to diagnose from thousands of cases a senior doctor has already labelled for you.
  • The golden rule — never evaluate on training data. That’s like boasting an A+ after memorising the past papers — it says nothing about how you’ll do on the real exam.
02 / Landscape

The ML Landscape

  • Programming vs statistics vs ML — programming applies hand-written rules (“cholesterol > 240 ⇒ refer”); statistics infers relationships to test a hypothesis; ML fits patterns to predict new cases. The borders blur, but the intent differs.
  • Supervised — labelled examples (X → y). Like a child seeing 5,000 labelled cats and dogs, then naming the next one. Today’s focus: classification and regression.
  • Unsupervised — no labels; find structure. Your playlist’s “Discover Weekly” groups similar songs; we cluster patients into subtypes whose phenotypes we didn’t define in advance.
  • Reinforcement — learn by trial-and-error with reward. How a Tetris bot, a chess engine, or your map app learns a better route over time. Used in adaptive trial design — not today.
  • Deep learning is a family of models, not a separate paradigm; it shines on images, text, signals (think radiology reads, Siri, ChatGPT) — we’ll save it for another day.
3
learning paradigms: supervised, unsupervised, reinforcement
2 tasks
we’ll cover today: classify a label, predict a number
03 / Workflow

The End-to-End ML Workflow

🎯
01 · Frame
A clinical question, a measurable target, a definition of “good enough”.
📥
02 · Data
Source, consent, representativeness. Garbage in, gospel out — still garbage.
🧬
03 · Features
Encode, scale, engineer. What the model actually sees.
✂️
04 · Split
Train / validation / test before anything else. No leakage.
🏋️
05 · Train
Fit one or several models. Compare, not assume.
📈
06 · Evaluate
Right metric for the clinical cost of each error type.
🚀
07 · Deploy
Silent in a clinician’s workflow until the loop closes.
🩺
08 · Monitor
Watch for drift; the world the model learned from changes.
“The model is the smallest part. Most ML projects fail at steps 01–03 and 07–08, not at the algorithm.”
03b / Daily Life

ML is Already All Around You

📧
Spam Filter
A classifier trained on millions of spam vs not-spam emails. Sensitive features: certain words, sender patterns. You’ve been using supervised ML for years.
🎬
Recommendations
Netflix groups users by taste (clustering = unsupervised), then predicts if you’ll like a new show (regression on predicted rating = supervised). Same math, different target.
🗺️
GPS ETA
Your map’s arrival time is a regression model: it predicts minutes-from-now using traffic, road type, weather, and historical patterns. Updates each second.
💳
Fraud Alerts
An anomaly-detection model flags “transaction unlike your usual”. Same idea as spotting a rare adverse drug reaction in pharmacovigilance data.
Pathology slides, radiology scans, ICU early-warning scores — the same algorithms. Medicine just demands them be safer.
03c / Learning

How a Model Learns

  • The loop — guess → measure how wrong → nudge the knobs → repeat. That’s all training is. The data doesn’t change; the model’s internal numbers do.
  • The knobs — called parameters. A logistic model has one weight per feature. More features, more knobs.
  • “How wrong” = the loss function — a single number. Smaller is better. Training minimises it. For regression that’s the squared error; for classification it’s usually log-loss (penalise overconfident wrong answers harder).
  • The nudge = gradient descent — walk downhill on the loss surface. Walk too far and you overshoot the valley; too slow and you never arrive. The step size is the learning rate.
  • The cooking analogy — you taste, add salt, taste again, adjust. You never taste the whole pot at once — you sample. A model never “sees” the truth, only how its guesses line up with the labels you gave it.
Loss ↓
the single number training minimises — lower is better
Knobs
parameters are learned; hyperparameters (depth, rate) are chosen by you
Stop
when loss stops falling — or you’ve converged (another word for “no further improvement”)
04 / scikit-learn

scikit-learn — The ML Workhorse

  • One API pattern — every model follows fit() → predict()
  • Dozens of algorithms — logistic regression, decision trees, random forests, SVM, k-NN
  • Built-in evaluation — accuracy, precision, recall, ROC curves, confusion matrices
  • Preprocessing tools — scaling, encoding, imputation, feature selection
# The 4-step recipe
from sklearn import SomeModel

model = SomeModel()
model.fit(X_train, y_train)
preds = model.predict(X_test)
04b / Features

Feature Engineering & Preprocessing

📏
Scaling
Put features on the same ruler. Glucose (90–300) and age (0–100) live on different scales; many algorithms need them comparable. Like converting inches and cm to the same unit before averaging.
🏷️
Encoding
Categories → numbers. One-hot blood type {A,B,O,AB} into four 0/1 columns; ordinal for cancer stage I–IV where order matters.
🧮
Missing Data
Drop, fill (impute), or model it. Always ask why it’s missing — “no BP recorded” in a critically ill patient is itself a clinical signal.
🧠
Engineering
Make the model’s job easier. BMI = weight/height², “days since admission”, “past-year admissions”. Domain knowledge beats raw columns almost every time.
Fit every preprocessing step on the training set only, then apply to validation/test. If you scale using the test set, the test has leaked into training — the most common form of data leakage.
05 / Splitting

Train · Validation · Test — Three Sets

  • Training set — the model learns here, like a student doing past papers with the answer key. Typically 60–70%.
  • Validation set — practice exams without the key. Used to tune hyperparameters and pick between models. Never used to fit weights.
  • Test set — the final exam, touched once, at the very end, to estimate real-world performance. Like a sealed envelope.
  • Parameters vs hyperparameters — parameters are learned from data (weights, thresholds); hyperparameters are chosen by you (tree depth, number of trees, regularisation). The student learns facts; you decide how many hours they study.
  • The cardinal sin — tuning on the test set. It silently becomes a second training set and your reported accuracy is fiction. The student who memorises the final exam hasn’t learned anything.
from sklearn.model_selection import train_test_split

# 60 / 20 / 20 in two cuts
X_tr, X_tmp, y_tr, y_tmp = train_test_split(
  X, y, test_size=0.4, stratify=y)
X_val, X_te, y_val, y_te = train_test_split(
  X_tmp, y_tmp, test_size=0.5, stratify=y_tmp)

# tune on val, judge on te — once
06 / Classification

Logistic Regression — Your First Model

  • Despite the name — logistic regression is a classification algorithm
  • Outputs a probability — "89% chance this tumor is malignant"
  • Fast, interpretable, and often excellent — always try it first
  • Dataset: Breast Cancer Wisconsin — 569 samples, 30 cell-nuclei features → predict malignant vs benign
07 / Intuition

Inside Logistic Regression

  • The idea — fit a linear combination of features, then squash it through the logistic (sigmoid) function to land in (0, 1) — a probability.
  • Decision boundary — a hyperplane where P = 0.5. Move the threshold up to be more conservative, down to be more sensitive.
  • Interpretable coefficients — each weight β reads as: “a one-unit rise in X multiplies the odds by eβ.” Clinicians like this.
  • Assumes linearity of log-odds — simple, transparent, but it cannot learn a curvy boundary by itself. That’s what trees and forests are for.
  • Calibration matters — “80% chance” should mean 8 out of 10 such patients truly have the condition. We’ll revisit this.
# linear score → probability
z = β₀ + β₁·glucose + β₂·age
p = 1 / (1 + e−z)

# choose a threshold
predict “malignant” if p > 0.5  # default
predict “malignant” if p > 0.2  # screen (high recall)
08 / Evaluation

Is Our Model Any Good?

🎯
Confusion Matrix
Shows exactly where errors happen. False negatives (missed cancers) vs false positives (false alarms).
📈
ROC Curve & AUC
Tradeoff between sensitivity and specificity at every threshold. AUC = 1.0 is perfect; 0.5 is random.
⚖️
Precision & Recall
Precision: of predicted positives, how many are real?
Recall: of real positives, how many did we catch?
🔬
Clinical Context
In cancer screening, recall > precision. Missing a cancer (false negative) is far worse than a false alarm.
08b / Accuracy Trap

Why Accuracy Lies

  • The fire-alarm trick — in a city where fires happen on 1% of buildings a year, an alarm that never rings is 99% accurate and 100% useless. Accuracy alone flatters a lazy model.
  • Class imbalance — 99 benign, 1 malignant: a model that always says “benign” scores 99%. A model that gets 98% looks worse on paper but is saving lives in reality.
  • Better baselines — report balanced accuracy, F1, or the ROC–AUC. They weight the two error types, not just the head-count.
  • Daily example — autocorrect rarely “corrects” anything (low base rate of typos), so a high accuracy is unimpressive. What matters is how often when you do type wrong it puts the right word on top — that’s precision at K.
  • Rule of thumb — never report a single accuracy number in isolation. State the prevalence, the metric, and the cost of each error.
99%
accuracy of an alarm that never rings — clinically useless
F1
harmonic mean of precision & recall — can’t be fooled by “always no”
09 / 2×2

Reading the Confusion Matrix

TP · True Positive

Cancer, called cancer. The win.

FP · False Positive

Healthy, called cancer. A smoke alarm that screams when there’s only toast — a scare, a biopsy, anxiety.

FN · False Negative

Cancer, called healthy. The alarm that stays silent while the kitchen burns — the dangerous error.

TN · True Negative

Healthy, called healthy. The other win.

  • Sensitivity (recall) = TP / (TP + FN). Out of all diseased, how many we catch.
  • Specificity = TN / (TN + FP). Out of all healthy, how many we correctly reassure.
  • PPV (precision) = TP / (TP + FP). Out of all called positive, how many truly are.
  • PPV depends on prevalence — even a 99% sensitive test gives a low PPV in a rare disease. This is why screening and confirmatory tests are different jobs.
  • Pick the metric for the cost you can’t afford — in screening, protect sensitivity; in confirmation, protect PPV.
10 / Overfitting

Overfitting vs Underfitting

📉
Underfitting
Model is too simple — like a student who only learnt the chapter titles. Misses real patterns; high bias, poor on train AND test.
🎯
Just Right
Learns genuine patterns without memorising noise. Like a student who understood the concepts — does well on questions never seen before.
⚠️
Overfitting
Model is too complex — memorised every past paper, including the typos. Great on training, terrible on test. The most common ML mistake.
💊
In Medicine
An overfit model might learn that "patients with ID #47 all have cancer" — a coincidence, not a pattern. Devastating in clinical use.
11 / Tradeoff

The Bias–Variance Tradeoff

  • Bias — error from a model too simple to capture the truth. A straight line through a curved world. (Darts analogy: all darts land in the top-left, far from bullseye — you have a consistent but wrong throw.)
  • Variance — error from a model too sensitive to the training sample. Two more patients and the answer swings. (Darts: clustered tightly, but in a different corner each round — unstable.)
  • Test error is U-shaped — starts high (underfit), falls to a sweet spot, climbs again (overfit). The job is to find the valley, not the most complex model.
  • Levers — add capacity (deeper trees, more features), or constrain it (regularisation, max depth, more data). You can’t lower both for free.
  • Ensembles change the curve — averaging many high-variance trees (a forest) keeps low bias and trims variance. Why forests often win.
High bias
underfit — training and test error both high
Sweet spot
balanced — test error at its minimum
High variance
overfit — training ≈ 0, test error climbing
12 / Trees

Decision Trees & Random Forest

  • Decision Tree — flowchart-like model; splits data by asking yes/no questions. You already play this: “20 Questions” — “Is it bigger than a car?” “Does it have fur?” Each split carves the possibilities until you guess. Fully interpretable.
  • Random Forest — an ensemble of 100+ decision trees, each trained on a random subset, then votes. Like asking 100 doctors the same case and taking the majority opinion: individual quirks wash out → more robust.
  • Feature importance — forests tell you which questions mattered most. The single most useful output for biomedical insight: it tells you which biomarkers to investigate.
  • Why doctors like trees — you can follow a decision tree’s logic by hand. “If glucose > 11 and age > 55 ⇒ high risk.” That auditability matters when a patient asks “why?”
from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(
  n_estimators=100,
  max_depth=5
)
rf.fit(X_train, y_train)

# Which features matter?
rf.feature_importances_
13 / Algorithms

The ML Toolbox

📏
k-Nearest Neighbours
“Show me the k most similar patients and take a vote.” No training, but distances break with many features.
📈
SVM
Finds the widest gap between classes. Powerful with kernels, but opaque and slow on big data.
🌳
Gradient Boosting
Trees built in sequence, each fixing the last’s mistakes. Often the accuracy leader; needs careful tuning.
🧠
Neural Nets
Layered features. Excellent for images, text, signals; needs more data, compute, and caution.
“No free lunch” — no algorithm wins on every problem. Start simple (logistic / tree), establish a baseline, then go complex only if it earns its keep.
14 / Validation

Cross-Validation — Trust but Verify

  • One split isn't enough — a single train/test split can be lucky (or unlucky). Cross-validation fixes this.
  • K-fold CV — split data into K parts. Train on K-1, test on 1. Repeat K times. Average the scores.
  • Typical K = 5 or 10 — gives a robust estimate of real-world performance.
  • In medicine — always report CV scores, not just one split. It shows whether your model is stable.
from sklearn.model_selection import cross_val_score

rf = RandomForestClassifier()
scores = cross_val_score(
  rf, X, y, cv=5
)

# 5 scores, one per fold
scores # eg [0.96, 0.98, 0.95, ...]
scores.mean() # robust estimate
15 / Tuning

Model Selection & Hyperparameters

  • What you choose — the algorithm and its settings: tree depth, number of trees, learning rate, regularisation strength, k in k-NN.
  • Grid search — try every combination on a grid, score each with CV. Thorough but expensive (blows up combinatorially).
  • Random search — sample combos at random. Surprisingly competitive — often finds near-optimal settings in a fraction of the tries.
  • Nested CV — when you also need an unbiased performance estimate, nest the tuning CV inside an outer evaluation CV.
  • Beware the leaderboard — tuning on the test set (or many rounds of it) is just overfitting to the test set. Hold it sacred.
3×3
a 3-value grid across 3 hyperparameters = 27 fits per CV fold
+CV
five-fold CV → 135 model fits. Tune thoughtfully, not exhaustively.
16 / Regression

Linear Regression — Predict a Number

  • Same idea, new target — linear regression is what your property appraiser uses to value a house from square footage, location, age. We use it to predict disease progression after one year from 10 baseline variables. The math is identical; only the units change.
  • Dataset: Diabetes — 442 patients, 10 baseline variables → predict disease progression after 1 year.
  • Error metrics — MAE (mean absolute error, same units as the target — “off by 12 points”), MSE / RMSE (penalise large errors more — one big miss hurts), R² (fraction of variance explained: 0 = worse than just averaging, 1 = perfect).
  • Coefficients — each feature has a weight. Positive = raises prediction, negative = lowers it. Standardise features first or weights aren’t comparable: a “1 mmHg” change isn’t the same scale as a “1 mg/dL” change.
  • Residuals tell the truth — plot predicted vs residual. A random scatter means the model is honest; a curve or fan-shape means it missed a pattern — the model is telling you where it’s still wrong.
  • Clinical insight — BMI and blood pressure are the strongest predictors. This matches clinical knowledge → builds trust, and flags BMI as a modifiable risk factor worth counselling on.
17 / Unsupervised

Unsupervised Learning

  • No labels, find structure — cluster, compress, visualise. Useful when labelling is expensive or the subtypes are unknown.
  • k-Means — partition patients into k groups by similarity. Try it on patient features to discover phenotypes you didn’t define in advance.
  • Hierarchical clustering — builds a tree of similarities; great for heatmaps of gene expression.
  • PCA — compress 100 correlated features into a handful of components that still capture most variance, for visualisation and noise reduction.
  • Anomaly detection — flag the unusual. Rare-event surveillance, fraud, rare adverse drug reactions.
  • Interpret with care — clusters and components are mathematical objects; the clinical meaning is a hypothesis to validate, not a finding.
k
chosen by you, not learned — pick by silhouette or domain sense, not by guesswork
PCA
components are linear mixes of original features — axes with no clinical label by default
18 / Datasets

Our Biomedical Datasets

🔬
Breast Cancer Wisconsin
569 samples, 30 features from cell nuclei images. Predict malignant vs benign. The classic biomedical ML benchmark.
💉
Diabetes Dataset
442 patients, 10 baseline variables (BMI, BP, blood serum). Predict disease progression 1 year later. Real clinical data.
📦
Built into scikit-learn
Both datasets come pre-installed with scikit-learn. One line of code loads them — no downloading, no cleaning needed.
🏥
Why These Matter
These are the same datasets used in ML textbooks and research papers worldwide. You're learning on real, published data.
19 / Insight

Model Interpretation & Common Pitfalls

  • Feature importance — random forests tell you which biomarkers matter most. This is often more valuable than the prediction itself.
  • Coefficients — in logistic/linear regression, each coefficient says: "for every 1-unit increase in X, outcome changes by Y"
  • Data leakage — accidentally using future information to predict the past. Example: scaling BEFORE splitting.
  • Class imbalance — 99% benign, 1% malignant? A model that says "always benign" gets 99% accuracy but is clinically useless.
  • Correlation ≠ causation — the model found that ice cream sales predict drownings. Doesn't mean ice cream causes drowning.
  • Always ask: Would I trust this model with a patient's life? If not, what would make you trust it?
20 / Safety

Clinical Validation & Safety

  • External validation — test on a dataset from a different hospital, era, or population. A model that only works where it trained has not learned medicine, it has learned a site.
  • Calibration — predicted probabilities must match observed frequencies. A 90% prediction should mean 9 out of 10. Overconfident models are dangerous models.
  • Dataset shift — the world drifts: equipment changes, demographics shift, protocols evolve. Re-evaluate periodically; a model is a perishable asset.
  • FDA “SaMD” — software as a medical device carries regulatory class I–III. The higher the risk, the higher the evidence bar.
  • Failure mode thinking — ask not “what is the accuracy?” but “who is harmed, and how badly, when it is wrong?”
~50%
of clinical prediction models degrade meaningfully on external data
1 / 1
test set, touched once — the only honest number you report
20b / Failure

When ML Fails in the Wild

  • The Amazon résumé tool (2018) — trained on 10 years of male-dominated hires, it quietly penalised women. Lesson: a model learns the biases in your history, by default.
  • Compas recidivism (US) — a risk-scoring tool was shown to be miscalibrated by race. Lesson: “equal accuracy” isn’t the same as “equal fairness”.
  • Sepsis alert deployed late — some hospital early-warning scores performed far worse after deployment than in their original papers. Lesson: external validation is not optional.
  • Self-driving edge cases — a sensor NULL in one cell of the input quietly meant “don’t trust the output when raining at dusk at an unusual crossing.” Lesson: rare cases fail silently.
  • Autocomplete & translation — gendered pronoun biases (“she is a nurse, he is a doctor”). Lesson: language inherits the world it was scraped from.
2×
error rate of a model after deployment vs. its published paper number
Why?
distribution shift, selection bias, feedback loops, gaming the target
“All models are wrong, but some are harmful.” The first question of deployment is never “how accurate?” — it is “who absorbs the cost of being wrong?”
21 / Ethics

Ethics, Bias & Fairness

👥
Representativeness
A model trained on one population can silently fail on another. Who is in your data — and who is missing?
⚖️
Algorithmic Bias
Models inherit historical bias: underdiagnosis of a group, unequal access, a feature that proxies for ethnicity or income.
🔓
Transparency
A clinician cannot act on a black box. Interpretability (SHAP, coefficients, calibrated probabilities) is a clinical requirement, not a nicety.
🤝
Accountability
A model advises; a clinician decides. Document the model, the data, the limits; keep a human in the loop for high-stakes calls.
“The most ethical model is not the most accurate one — it is the one whose failures you can predict, explain, and justify to the patient it failed.”
22 / Setup

How We'll Work

🌐
Google Colab
No installation. scikit-learn, NumPy, Pandas, and Matplotlib all pre-installed. Runs in your browser.
⌨️
Code-Along
I'll type, you follow. Every cell runs. You'll see models train in real time and predictions appear immediately.
✏️
Exercises
Build your own models. Compare algorithms. Decide which model you'd trust in a clinical setting.
💬
Ask Anytime
ML has jargon — if a term doesn't make sense, ask. Chances are others have the same question.
23 / Recap

The ML Pipeline

Split
Train/test before anything else
Fit
model.fit(X_train, y_train)
Evaluate
Confusion matrix, ROC, precision, recall
"Every ML pipeline is the same: split → fit → predict → evaluate. Master this pattern and you can use any algorithm."
🤖

Let's Build Models!

Open your notebook. Time to train your first classifier.

Open in Google Colab
Home 1 / 32 Download Notebook