{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conventional Machine Learning\n",
    "## A Hands-On Workshop for Biomedical Professionals\n",
    "\n",
    "**Course**: AI in Medicine — Summer Course 2026  \n",
    "**Date**: 28 July 2026 · 9:00–12:15  \n",
    "**Level**: Beginner to Intermediate  \n",
    "**Environment**: Google Colab  \n",
    "**Datasets**: Breast Cancer Wisconsin & Diabetes (scikit-learn)\n",
    "\n",
    "---\n",
    "### What We'll Build Today\n",
    "\n",
    "| # | Section | Duration |\n",
    "|----|---------|----------|\n",
    "| 1 | What is Machine Learning? | ~15 min |\n",
    "| 2 | Classification I — Logistic Regression | ~30 min |\n",
    "| 3 | Model Evaluation | ~20 min |\n",
    "| 4 | Classification II — Decision Trees & Random Forest | ~25 min |\n",
    "| 5 | Regression | ~25 min |\n",
    "| 6 | Mini Project — Breast Cancer Classification | ~50 min |\n",
    "| 7 | Wrap-up & Next Steps | ~10 min |\n",
    "\n",
    "---\n",
    "### Why Machine Learning for Biomedicine?\n",
    "\n",
    "- **Diagnosis**: Classify tumors as malignant or benign from cell measurements\n",
    "- **Prognosis**: Predict disease progression from patient data\n",
    "- **Screening**: Identify high-risk patients from routine lab results\n",
    "- **Research**: Discover which biomarkers matter most for outcomes\n",
    "\n",
    "> Today you'll train models that make real medical predictions — and learn to evaluate whether they're trustworthy.\n",
    "\n",
    "---\n",
    "### Before We Start\n",
    "All libraries are pre-installed in Colab. Just run the imports below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# scikit-learn — the ML workhorse\n",
    "from sklearn.model_selection import train_test_split, cross_val_score\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.linear_model import LogisticRegression, LinearRegression\n",
    "from sklearn.tree import DecisionTreeClassifier, plot_tree\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import (accuracy_score, precision_score, recall_score, f1_score,\n",
    "                             confusion_matrix, classification_report, roc_curve, auc,\n",
    "                             mean_absolute_error, mean_squared_error, r2_score)\n",
    "from sklearn.datasets import load_breast_cancer, load_diabetes\n",
    "\n",
    "sns.set_style('whitegrid')\n",
    "%matplotlib inline\n",
    "\n",
    "print('scikit-learn:', __import__('sklearn').__version__)\n",
    "print('All libraries loaded!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 1: What is Machine Learning?\n",
    "\n",
    "**Machine Learning** = algorithms that learn patterns from data, then make predictions on new data.\n",
    "\n",
    "### Two Main Types\n",
    "\n",
    "| Type | What it does | Biomedical example |\n",
    "|------|-------------|-------------------|\n",
    "| **Classification** | Predict a category | Malignant vs benign tumor |\n",
    "| **Regression** | Predict a number | Disease progression score |\n",
    "\n",
    "### The scikit-learn Recipe\n",
    "Almost every ML task in scikit-learn follows the same 4-step pattern:\n",
    "\n",
    "```python\n",
    "# 1. Import the model\n",
    "from sklearn.some_module import SomeModel\n",
    "\n",
    "# 2. Create an instance\n",
    "model = SomeModel()\n",
    "\n",
    "# 3. Train (fit) on data\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "# 4. Predict on new data\n",
    "predictions = model.predict(X_test)\n",
    "```\n",
    "\n",
    "### Train/Test Split — The Golden Rule\n",
    "**Never evaluate a model on the same data it was trained on.** Always hold out a test set.\n",
    "\n",
    "```python\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "```\n",
    "\n",
    "> 🎯 **Key insight**: ML isn't magic. It's pattern recognition at scale. If there's no pattern in your data, no algorithm can find one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 2: Classification I — Logistic Regression\n",
    "\n",
    "Despite the name, logistic regression is a **classification** algorithm. It estimates the probability that a sample belongs to a class.\n",
    "\n",
    "**Our dataset**: [Breast Cancer Wisconsin](https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic) — 569 samples, 30 features from cell nuclei images. Goal: predict malignant vs benign.\n",
    "\n",
    "### Load & Explore the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load breast cancer dataset\n",
    "cancer = load_breast_cancer()\n",
    "X = cancer.data\n",
    "y = cancer.target  # 0 = malignant, 1 = benign\n",
    "\n",
    "print(f'Samples:  {X.shape[0]}')\n",
    "print(f'Features: {X.shape[1]}')\n",
    "print(f'Classes:  {cancer.target_names}')\n",
    "print(f'\\nMalignant (0): {np.sum(y == 0)} samples')\n",
    "print(f'Benign (1):    {np.sum(y == 1)} samples')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert to DataFrame for easier exploration\n",
    "df_cancer = pd.DataFrame(X, columns=cancer.feature_names)\n",
    "df_cancer['target'] = y\n",
    "df_cancer['diagnosis'] = df_cancer['target'].map({0: 'Malignant', 1: 'Benign'})\n",
    "\n",
    "print('First 5 samples:')\n",
    "df_cancer[['diagnosis'] + list(cancer.feature_names[:5])].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Look at one feature: mean radius\n",
    "fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
    "\n",
    "axes[0].hist(df_cancer[df_cancer['diagnosis'] == 'Malignant']['mean radius'],\n",
    "             bins=20, alpha=0.7, color='coral', label='Malignant')\n",
    "axes[0].hist(df_cancer[df_cancer['diagnosis'] == 'Benign']['mean radius'],\n",
    "             bins=20, alpha=0.7, color='steelblue', label='Benign')\n",
    "axes[0].set_title('Mean Radius by Diagnosis')\n",
    "axes[0].set_xlabel('Mean Radius')\n",
    "axes[0].set_ylabel('Count')\n",
    "axes[0].legend()\n",
    "\n",
    "# Box plot of several features\n",
    "features_to_plot = ['mean radius', 'mean texture', 'mean perimeter', 'mean area']\n",
    "df_melted = df_cancer.melt(id_vars=['diagnosis'], value_vars=features_to_plot,\n",
    "                            var_name='Feature', value_name='Value')\n",
    "sns.boxplot(data=df_melted, x='Feature', y='Value', hue='diagnosis',\n",
    "            palette={'Malignant': 'coral', 'Benign': 'steelblue'}, ax=axes[1])\n",
    "axes[1].set_title('Feature Distributions by Diagnosis')\n",
    "axes[1].tick_params(axis='x', rotation=30)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train/Test Split & Scale\n",
    "Always split BEFORE scaling to avoid data leakage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split data\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=42, stratify=y\n",
    ")\n",
    "\n",
    "print(f'Train set: {X_train.shape[0]} samples')\n",
    "print(f'Test set:  {X_test.shape[0]} samples')\n",
    "print(f'Train class balance: {np.mean(y_train):.1%} benign')\n",
    "print(f'Test  class balance: {np.mean(y_test):.1%} benign')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale features (important for logistic regression)\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)  # fit ONLY on train\n",
    "X_test_scaled = scaler.transform(X_test)         # transform test with SAME scaler\n",
    "\n",
    "print('Scaling complete — all features now have mean≈0, std≈1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The 4-step scikit-learn recipe\n",
    "logreg = LogisticRegression(max_iter=1000, random_state=42)\n",
    "logreg.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Predict on test set\n",
    "y_pred = logreg.predict(X_test_scaled)\n",
    "y_prob = logreg.predict_proba(X_test_scaled)[:, 1]  # probability of benign\n",
    "\n",
    "# Quick accuracy check\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "print(f'Accuracy: {acc:.2%}')\n",
    "print(f'Correctly classified: {np.sum(y_pred == y_test)} out of {len(y_test)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confusion Matrix\n",
    "Accuracy alone can be misleading. The confusion matrix shows where errors happen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cm = confusion_matrix(y_test, y_pred)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6, 5))\n",
    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=ax,\n",
    "            xticklabels=['Malignant', 'Benign'],\n",
    "            yticklabels=['Malignant', 'Benign'])\n",
    "ax.set_xlabel('Predicted')\n",
    "ax.set_ylabel('Actual')\n",
    "ax.set_title('Confusion Matrix — Logistic Regression', fontweight='bold')\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(classification_report(y_test, y_pred, target_names=cancer.target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🤔 **Think**: A false negative (predicting benign when it's malignant) is far worse than a false positive in cancer diagnosis. This is why accuracy alone isn't enough — we care about **recall** (sensitivity).\n",
    "\n",
    "> 🎯 **Classification Takeaway**: Logistic regression is simple, fast, and interpretable. With scaling, it's often competitive with more complex models — always try it first."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 3: Model Evaluation\n",
    "\n",
    "How good is our model? Accuracy tells part of the story. Let's look at the full picture.\n",
    "\n",
    "### Key Metrics\n",
    "\n",
    "| Metric | Formula | What it means |\n",
    "|--------|---------|--------------|\n",
    "| **Accuracy** | (TP+TN) / Total | How often correct overall |\n",
    "| **Precision** | TP / (TP+FP) | Of predicted positives, how many are real? |\n",
    "| **Recall (Sensitivity)** | TP / (TP+FN) | Of real positives, how many did we catch? |\n",
    "| **F1 Score** | 2×P×R / (P+R) | Harmonic mean of precision & recall |\n",
    "\n",
    "> In cancer screening: high recall (catch every case) > high precision (avoid false alarms). In drug toxicity screening: the opposite."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate all metrics\n",
    "precision = precision_score(y_test, y_pred)\n",
    "recall = recall_score(y_test, y_pred)\n",
    "f1 = f1_score(y_test, y_pred)\n",
    "\n",
    "print(f'Accuracy:  {accuracy_score(y_test, y_pred):.3f}')\n",
    "print(f'Precision: {precision:.3f}')\n",
    "print(f'Recall:    {recall:.3f}  (sensitivity)')\n",
    "print(f'F1 Score:  {f1:.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ROC Curve & AUC\n",
    "Shows the tradeoff between true positive rate and false positive rate at every threshold."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ROC curve\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_prob)\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 6))\n",
    "ax.plot(fpr, tpr, color='steelblue', linewidth=2.5, label=f'ROC (AUC = {roc_auc:.3f})')\n",
    "ax.plot([0, 1], [0, 1], '--', color='gray', linewidth=1, label='Random (AUC = 0.5)')\n",
    "ax.fill_between(fpr, tpr, alpha=0.15, color='steelblue')\n",
    "ax.set_xlabel('False Positive Rate (1 - Specificity)')\n",
    "ax.set_ylabel('True Positive Rate (Sensitivity)')\n",
    "ax.set_title('ROC Curve — Logistic Regression', fontweight='bold')\n",
    "ax.legend(loc='lower right')\n",
    "ax.set_xlim([0, 1]); ax.set_ylim([0, 1])\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(f'AUC = {roc_auc:.3f} → {\"Excellent\" if roc_auc > 0.9 else \"Good\" if roc_auc > 0.8 else \"Fair\"} discrimination')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ✏️ Your Turn (5 min)\n",
    "Run logistic regression WITHOUT scaling (use `X_train` and `X_test` directly). Compare the accuracy and AUC to the scaled version. Is scaling important for this model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your experiment here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 4: Overfitting & Cross-Validation\n",
    "\n",
    "A model that memorizes the training data perfectly — but fails on new data — is **overfit**. This is the most common ML mistake.\n",
    "\n",
    "### The Telltale Sign\n",
    "Training accuracy keeps improving, but test accuracy plateaus or drops. The model is learning noise, not signal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Demo: varying max_depth shows overfitting in action\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "depths = range(1, 21)\n",
    "train_scores = []\n",
    "test_scores = []\n",
    "\n",
    "for depth in depths:\n",
    "    tree = DecisionTreeClassifier(max_depth=depth, random_state=42)\n",
    "    tree.fit(X_train_scaled, y_train)\n",
    "    train_scores.append(tree.score(X_train_scaled, y_train))\n",
    "    test_scores.append(tree.score(X_test_scaled, y_test))\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "ax.plot(depths, train_scores, 'o-', color='steelblue', linewidth=2, label='Training Accuracy')\n",
    "ax.plot(depths, test_scores, 's-', color='coral', linewidth=2, label='Test Accuracy')\n",
    "ax.axvline(depths[np.argmax(test_scores)], color='gray', linestyle='--',\n",
    "           label=f'Best depth = {depths[np.argmax(test_scores)]}')\n",
    "ax.set_xlabel('max_depth')\n",
    "ax.set_ylabel('Accuracy')\n",
    "ax.set_title('Overfitting Demo — Decision Tree Depth vs Accuracy', fontweight='bold')\n",
    "ax.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(f'Best max_depth: {depths[np.argmax(test_scores)]}')\n",
    "print(f'Training accuracy at depth 20: {train_scores[-1]:.3f} (perfect! but...)')\n",
    "print(f'Test accuracy at depth 20:     {test_scores[-1]:.3f} (worse than depth 3!)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🤔 **The gap between train and test accuracy is the overfitting signal.** When training = 1.0 and test = 0.92, your model has memorized — it hasn't learned.\n",
    "\n",
    "### Cross-Validation — The Cure for Lucky Splits\n",
    "A single train/test split can be lucky. Cross-validation splits the data K times and averages the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 5-fold cross-validation\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "logreg = LogisticRegression(max_iter=2000, random_state=42)\n",
    "cv_scores = cross_val_score(logreg, X_train_scaled, y_train, cv=5)\n",
    "\n",
    "print('5-fold CV scores:')\n",
    "for i, score in enumerate(cv_scores, 1):\n",
    "    print(f'  Fold {i}: {score:.4f}')\n",
    "print(f'\\nMean CV Accuracy: {cv_scores.mean():.4f}')\n",
    "print(f'Std CV Accuracy:  {cv_scores.std():.4f}  (lower = more stable)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compare models with cross-validation\n",
    "models_for_cv = {\n",
    "    'Logistic Regression': LogisticRegression(max_iter=2000, random_state=42),\n",
    "    'Decision Tree (depth=3)': DecisionTreeClassifier(max_depth=3, random_state=42),\n",
    "    'Decision Tree (depth=10)': DecisionTreeClassifier(max_depth=10, random_state=42),\n",
    "    'Random Forest (depth=5)': RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)\n",
    "}\n",
    "\n",
    "for name, model in models_for_cv.items():\n",
    "    scores = cross_val_score(model, X_train_scaled, y_train, cv=5)\n",
    "    print(f'{name:30s}: {scores.mean():.4f} (+/- {scores.std():.4f})')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🎯 **Overfitting Takeaway**: Watch the train-test gap. Use cross-validation, not a single split. In medicine, an overfit model can kill — it looks great on paper, fails in the clinic."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 5: Classification II — Decision Trees & Random Forest\n",
    "\n",
    "### Decision Tree\n",
    "A flowchart-like model that splits data by asking yes/no questions about features. Highly interpretable — you can literally draw the tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a decision tree\n",
    "dt = DecisionTreeClassifier(max_depth=3, random_state=42)  # shallow tree = interpretable\n",
    "dt.fit(X_train_scaled, y_train)\n",
    "\n",
    "y_pred_dt = dt.predict(X_test_scaled)\n",
    "print(f'Decision Tree Accuracy: {accuracy_score(y_test, y_pred_dt):.3f}')\n",
    "print(f'\\nClassification Report:')\n",
    "print(classification_report(y_test, y_pred_dt, target_names=cancer.target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize the tree\n",
    "fig, ax = plt.subplots(figsize=(16, 8))\n",
    "plot_tree(dt, feature_names=cancer.feature_names, class_names=list(cancer.target_names),\n",
    "          filled=True, rounded=True, fontsize=9, ax=ax)\n",
    "ax.set_title('Decision Tree (max_depth=3) — Breast Cancer Diagnosis', fontsize=16, fontweight='bold')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🤔 **Look at the tree**: The first split is on `worst perimeter`. This means that of all 30 features, the model chose this as the single best question to separate malignant from benign. This is **feature importance** in action."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest\n",
    "An ensemble of many decision trees, each trained on a random subset of data. Averages their predictions — more robust, less prone to overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a random forest\n",
    "rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)\n",
    "rf.fit(X_train_scaled, y_train)\n",
    "\n",
    "y_pred_rf = rf.predict(X_test_scaled)\n",
    "y_prob_rf = rf.predict_proba(X_test_scaled)[:, 1]\n",
    "\n",
    "print(f'Random Forest Accuracy: {accuracy_score(y_test, y_pred_rf):.3f}')\n",
    "print(classification_report(y_test, y_pred_rf, target_names=cancer.target_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Feature importance — which features matter most?\n",
    "importances = rf.feature_importances_\n",
    "indices = np.argsort(importances)[-10:]  # top 10\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "ax.barh(range(10), importances[indices], color='steelblue', edgecolor='white')\n",
    "ax.set_yticks(range(10))\n",
    "ax.set_yticklabels([cancer.feature_names[i] for i in indices])\n",
    "ax.set_xlabel('Importance')\n",
    "ax.set_title('Top 10 Features — Random Forest', fontweight='bold')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compare all three models\n",
    "models = {\n",
    "    'Logistic Regression': LogisticRegression(max_iter=1000, random_state=42),\n",
    "    'Decision Tree': DecisionTreeClassifier(max_depth=5, random_state=42),\n",
    "    'Random Forest': RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)\n",
    "}\n",
    "\n",
    "results = []\n",
    "for name, model in models.items():\n",
    "    model.fit(X_train_scaled, y_train)\n",
    "    y_pred_m = model.predict(X_test_scaled)\n",
    "    y_prob_m = model.predict_proba(X_test_scaled)[:, 1] if hasattr(model, 'predict_proba') else None\n",
    "    results.append({\n",
    "        'Model': name,\n",
    "        'Accuracy': accuracy_score(y_test, y_pred_m),\n",
    "        'Precision': precision_score(y_test, y_pred_m),\n",
    "        'Recall': recall_score(y_test, y_pred_m),\n",
    "        'F1': f1_score(y_test, y_pred_m),\n",
    "        'AUC': auc(roc_curve(y_test, y_prob_m)[0], roc_curve(y_test, y_prob_m)[1]) if y_prob_m is not None else np.nan\n",
    "    })\n",
    "\n",
    "df_results = pd.DataFrame(results).set_index('Model')\n",
    "df_results.style.background_gradient(cmap='Blues', axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🎯 **Takeaway**: Random Forest usually outperforms a single decision tree. But logistic regression is often close — and far more interpretable. In medicine, **explainability matters**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 5: Regression\n",
    "\n",
    "Regression predicts a **continuous number** — not a category.\n",
    "\n",
    "**Our dataset**: [Diabetes](https://scikit-learn.org/stable/datasets/toy_dataset.html#diabetes-dataset) — 442 patients, 10 baseline variables, target = disease progression 1 year later.\n",
    "\n",
    "### Load & Explore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load diabetes dataset\n",
    "diabetes = load_diabetes()\n",
    "X_dia = diabetes.data\n",
    "y_dia = diabetes.target\n",
    "\n",
    "df_dia = pd.DataFrame(X_dia, columns=diabetes.feature_names)\n",
    "df_dia['progression'] = y_dia\n",
    "\n",
    "print(f'Samples: {X_dia.shape[0]}')\n",
    "print(f'Features: {X_dia.shape[1]}')\n",
    "print(f'Target range: {y_dia.min():.0f} to {y_dia.max():.0f}')\n",
    "print(f'Target mean:  {y_dia.mean():.0f}')\n",
    "df_dia.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize the target\n",
    "fig, ax = plt.subplots(figsize=(8, 4))\n",
    "ax.hist(y_dia, bins=30, color='steelblue', edgecolor='white', alpha=0.85)\n",
    "ax.axvline(y_dia.mean(), color='coral', linestyle='--', linewidth=2, label=f'Mean = {y_dia.mean():.0f}')\n",
    "ax.set_xlabel('Disease Progression (1 year)')\n",
    "ax.set_ylabel('Number of Patients')\n",
    "ax.set_title('Diabetes Disease Progression — Target Distribution', fontweight='bold')\n",
    "ax.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split & scale\n",
    "X_train_d, X_test_d, y_train_d, y_test_d = train_test_split(\n",
    "    X_dia, y_dia, test_size=0.2, random_state=42\n",
    ")\n",
    "\n",
    "scaler_dia = StandardScaler()\n",
    "X_train_d_s = scaler_dia.fit_transform(X_train_d)\n",
    "X_test_d_s = scaler_dia.transform(X_test_d)\n",
    "\n",
    "# Train\n",
    "lr = LinearRegression()\n",
    "lr.fit(X_train_d_s, y_train_d)\n",
    "y_pred_d = lr.predict(X_test_d_s)\n",
    "\n",
    "# Evaluate\n",
    "print(f'R-squared (R²): {r2_score(y_test_d, y_pred_d):.3f}')\n",
    "print(f'MAE:            {mean_absolute_error(y_test_d, y_pred_d):.1f}')\n",
    "print(f'RMSE:           {np.sqrt(mean_squared_error(y_test_d, y_pred_d)):.1f}')\n",
    "print(f'\\nTarget mean: {y_test_d.mean():.0f}, std: {y_test_d.std():.0f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Actual vs Predicted\n",
    "The gold-standard plot for regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(7, 6))\n",
    "ax.scatter(y_test_d, y_pred_d, alpha=0.6, color='steelblue', edgecolors='white')\n",
    "ax.plot([y_test_d.min(), y_test_d.max()], [y_test_d.min(), y_test_d.max()],\n",
    "        '--', color='coral', linewidth=2, label='Perfect Prediction')\n",
    "ax.set_xlabel('Actual Progression')\n",
    "ax.set_ylabel('Predicted Progression')\n",
    "ax.set_title('Actual vs Predicted — Linear Regression', fontweight='bold')\n",
    "ax.legend()\n",
    "ax.set_aspect('equal')\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Residuals\n",
    "residuals = y_test_d - y_pred_d\n",
    "print(f'Mean residual (bias): {residuals.mean():.1f} (should be ≈0)')\n",
    "print(f'Std of residuals:     {residuals.std():.1f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Importance in Linear Regression\n",
    "The coefficients tell you which features push the prediction up or down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Coefficients (already scaled, so comparable)\n",
    "coef_df = pd.DataFrame({\n",
    "    'Feature': diabetes.feature_names,\n",
    "    'Coefficient': lr.coef_\n",
    "}).sort_values('Coefficient', ascending=False)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "colors = ['coral' if c > 0 else 'steelblue' for c in coef_df['Coefficient']]\n",
    "ax.barh(coef_df['Feature'], coef_df['Coefficient'], color=colors, edgecolor='white')\n",
    "ax.axvline(0, color='black', linewidth=0.5)\n",
    "ax.set_xlabel('Coefficient (effect on progression)')\n",
    "ax.set_title('Linear Regression Coefficients — Which features matter most?', fontweight='bold')\n",
    "ax.invert_yaxis()\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print('Top features that INCREASE predicted progression:')\n",
    "print(coef_df.head(3).to_string(index=False))\n",
    "print('\\nFeatures that DECREASE predicted progression:')\n",
    "print(coef_df.tail(3).to_string(index=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🤔 **Clinical interpretation**: BMI and blood pressure (bp) are the strongest predictors of diabetes progression. This aligns with clinical knowledge — which builds trust in the model.\n",
    "\n",
    "> 🎯 **Regression Takeaway**: R² tells you what fraction of variance your model explains (0 to 1). R² near 1 = great. R² near 0 = your model is no better than guessing the mean. For the diabetes data, R²≈0.45 is typical — disease progression isn't fully predictable from these 10 variables alone."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 6: Mini Project — Breast Cancer Classification\n",
    "\n",
    "Now you'll build a complete ML pipeline — from raw data to model comparison — on the breast cancer dataset.\n",
    "\n",
    "**Your tasks**:\n",
    "1. Load and explore the breast cancer data\n",
    "2. Train at least 3 different models (logistic regression, decision tree, random forest)\n",
    "3. Compare them using accuracy, precision, recall, F1, and AUC\n",
    "4. Identify the most important features\n",
    "5. Answer: Which model would you deploy in a clinical setting? Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q1: Load & Split\n",
    "Load the breast cancer dataset, split into train/test (80/20, stratified), and scale the features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q1: Your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 💡 Solution\n",
    "cancer = load_breast_cancer()\n",
    "X, y = cancer.data, cancer.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=42, stratify=y\n",
    ")\n",
    "scaler = StandardScaler()\n",
    "X_train_s = scaler.fit_transform(X_train)\n",
    "X_test_s = scaler.transform(X_test)\n",
    "print(f'Train: {X_train_s.shape[0]}, Test: {X_test_s.shape[0]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q2: Train Three Models\n",
    "Train logistic regression, decision tree (max_depth=5), and random forest (100 trees, max_depth=5). Store predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q2: Your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 💡 Solution\n",
    "models = {\n",
    "    'Logistic Regression': LogisticRegression(max_iter=2000, random_state=42),\n",
    "    'Decision Tree': DecisionTreeClassifier(max_depth=5, random_state=42),\n",
    "    'Random Forest': RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)\n",
    "}\n",
    "\n",
    "predictions = {}\n",
    "probabilities = {}\n",
    "for name, model in models.items():\n",
    "    model.fit(X_train_s, y_train)\n",
    "    predictions[name] = model.predict(X_test_s)\n",
    "    probabilities[name] = model.predict_proba(X_test_s)[:, 1]\n",
    "    print(f'{name}: trained')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q3: Compare Performance\n",
    "Create a comparison table with accuracy, precision, recall, F1, and AUC for all three models. Which performs best?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q3: Your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 💡 Solution\n",
    "results = []\n",
    "for name in models:\n",
    "    y_pred = predictions[name]\n",
    "    y_prob = probabilities[name]\n",
    "    fpr, tpr, _ = roc_curve(y_test, y_prob)\n",
    "    results.append({\n",
    "        'Model': name,\n",
    "        'Accuracy': accuracy_score(y_test, y_pred),\n",
    "        'Precision': precision_score(y_test, y_pred),\n",
    "        'Recall': recall_score(y_test, y_pred),\n",
    "        'F1 Score': f1_score(y_test, y_pred),\n",
    "        'AUC': auc(fpr, tpr)\n",
    "    })\n",
    "\n",
    "df_compare = pd.DataFrame(results).set_index('Model')\n",
    "df_compare.style.background_gradient(cmap='Blues', axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q4: ROC Curves\n",
    "Plot all three ROC curves on the same chart."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q4: Your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 💡 Solution\n",
    "fig, ax = plt.subplots(figsize=(8, 7))\n",
    "colors = {'Logistic Regression': 'steelblue', 'Decision Tree': 'coral', 'Random Forest': 'green'}\n",
    "\n",
    "for name, model in models.items():\n",
    "    fpr, tpr, _ = roc_curve(y_test, probabilities[name])\n",
    "    roc_auc = auc(fpr, tpr)\n",
    "    ax.plot(fpr, tpr, linewidth=2.5, color=colors[name], label=f'{name} (AUC={roc_auc:.3f})')\n",
    "\n",
    "ax.plot([0, 1], [0, 1], '--', color='gray', linewidth=1, label='Random')\n",
    "ax.set_xlabel('False Positive Rate'); ax.set_ylabel('True Positive Rate')\n",
    "ax.set_title('ROC Curves — All Models', fontweight='bold')\n",
    "ax.legend(loc='lower right')\n",
    "ax.set_xlim([0, 1]); ax.set_ylim([0, 1])\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q5: Feature Importance\n",
    "Extract and plot the top 10 most important features from the random forest model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q5: Your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 💡 Solution\n",
    "rf_model = models['Random Forest']\n",
    "importances = rf_model.feature_importances_\n",
    "indices = np.argsort(importances)[-10:]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "ax.barh(range(10), importances[indices], color='steelblue', edgecolor='white')\n",
    "ax.set_yticks(range(10))\n",
    "ax.set_yticklabels([cancer.feature_names[i] for i in indices])\n",
    "ax.set_xlabel('Importance')\n",
    "ax.set_title('Top 10 Features — Random Forest', fontweight='bold')\n",
    "ax.invert_yaxis()\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q6: Clinical Decision\n",
    "\n",
    "**Answer in your own words**: Which model would you deploy in a clinical setting, and why? Consider:\n",
    "- Accuracy and ROC AUC\n",
    "- Interpretability (can a doctor understand why the model made a prediction?)\n",
    "- Speed (how fast does it predict?)\n",
    "- Which errors are more dangerous — false positives or false negatives?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Your answer here — double-click to edit*\n",
    "\n",
    "I would deploy **Logistic Regression** because...\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 🎯 **Mini Project Takeaway**: You just built, evaluated, and compared three ML models on a real biomedical dataset — and made a clinical deployment decision. This is what ML in medicine looks like."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 8: Common Pitfalls & Model Saving\n",
    "\n",
    "Before you deploy an ML model in a biomedical setting, know these traps — and how to avoid them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pitfall 1: Data Leakage\n",
    "Using information from the test set during training. Example: scaling before splitting — the scaler 'sees' the test data through the mean/std calculation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DON'T do this (data leakage):\n",
    "# scaler = StandardScaler()\n",
    "# X_all_scaled = scaler.fit_transform(X)\n",
    "# X_train, X_test, y_train, y_test = train_test_split(X_all_scaled, y)\n",
    "\n",
    "# DO this instead:\n",
    "# X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "# scaler = StandardScaler()\n",
    "# X_train_s = scaler.fit_transform(X_train)\n",
    "# X_test_s = scaler.transform(X_test)\n",
    "print('Always: split first, then fit scaler on train only')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pitfall 2: Class Imbalance\n",
    "If 99% of samples are benign, a model that always says 'benign' gets 99% accuracy — but catches zero cancers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check class balance\n",
    "print('Our breast cancer data is balanced:')\n",
    "print(f'  Malignant: {np.sum(cancer.target == 0)} ({np.mean(cancer.target == 0)*100:.0f}%)')\n",
    "print(f'  Benign:    {np.sum(cancer.target == 1)} ({np.mean(cancer.target == 1)*100:.0f}%)')\n",
    "print('\\nIf your data is imbalanced, use:')\n",
    "print('  - stratify=y in train_test_split')\n",
    "print('  - class_weight=\"balanced\" in the model')\n",
    "print('  - SMOTE for oversampling minority class')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pitfall 3: Correlation vs Causation\n",
    "The model finds patterns, not causes. A model might predict 'high cancer risk' from 'visited hospital frequently' — but the hospital visits are because of the cancer, not the cause.\n",
    "\n",
    "> **In medicine**: Always validate ML findings with domain expertise. The model suggests — the doctor decides."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving & Loading Models\n",
    "You've trained a model — now save it for later use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save a trained model\n",
    "import joblib\n",
    "\n",
    "# Train a final model on all training data\n",
    "final_model = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)\n",
    "final_model.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Save to disk\n",
    "joblib.dump(final_model, 'breast_cancer_model.joblib')\n",
    "joblib.dump(scaler, 'scaler.joblib')  # save the scaler too!\n",
    "print('Model and scaler saved!')\n",
    "\n",
    "# Later, load and predict\n",
    "loaded_model = joblib.load('breast_cancer_model.joblib')\n",
    "loaded_scaler = joblib.load('scaler.joblib')\n",
    "\n",
    "# Predict on new data (first 3 test samples)\n",
    "new_data = X_test[:3]\n",
    "new_data_scaled = loaded_scaler.transform(new_data)\n",
    "predictions = loaded_model.predict(new_data_scaled)\n",
    "print(f'Predictions for 3 new samples: {predictions}')\n",
    "print(f'Diagnosis: {[cancer.target_names[p] for p in predictions]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Pitfalls Takeaway: Split before scaling. Check class balance. Never confuse correlation with causation. Save your model and scaler together. A model you cannot reproduce is a model that doesn't exist."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Section 9: Wrap-Up\n",
    "\n",
    "### What You Learned Today\n",
    "\n",
    "| Skill | Tool | Biomedical Application |\n",
    "|-------|------|----------------------|\n",
    "| Train/test split | `train_test_split` | Fair evaluation of any model |\n",
    "| Feature scaling | `StandardScaler` | Required for linear models |\n",
    "| Logistic regression | `LogisticRegression` | Binary diagnosis (malignant/benign) |\n",
    "| Decision trees | `DecisionTreeClassifier` | Interpretable rules for triage |\n",
    "| Random forest | `RandomForestClassifier` | Robust, high-accuracy classification |\n",
    "| Confusion matrix | `confusion_matrix` | Understand where errors happen |\n",
    "| ROC/AUC | `roc_curve`, `auc` | Model discrimination quality |\n",
    "| Linear regression | `LinearRegression` | Predict disease progression |\n",
    "| Model comparison | Multiple metrics | Choose the right model for the job |\n",
    "\n",
    "### Key scikit-learn Cheat Sheet\n",
    "\n",
    "```python\n",
    "# Imports\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.linear_model import LogisticRegression, LinearRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score, classification_report, confusion_matrix\n",
    "from sklearn.metrics import roc_curve, auc, r2_score\n",
    "\n",
    "# Always split first\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)\n",
    "\n",
    "# Scale (for linear models)\n",
    "scaler = StandardScaler()\n",
    "X_train_s = scaler.fit_transform(X_train)\n",
    "X_test_s = scaler.transform(X_test)\n",
    "\n",
    "# Train\n",
    "model = LogisticRegression()\n",
    "model.fit(X_train_s, y_train)\n",
    "\n",
    "# Predict & evaluate\n",
    "y_pred = model.predict(X_test_s)\n",
    "print(classification_report(y_test, y_pred))\n",
    "print(confusion_matrix(y_test, y_pred))\n",
    "```\n",
    "\n",
    "### Where to Go From Here\n",
    "\n",
    "1. **More algorithms** — SVM, k-NN, Gradient Boosting (XGBoost)\n",
    "2. **Cross-validation** — `cross_val_score` for robust evaluation\n",
    "3. **Hyperparameter tuning** — `GridSearchCV` to find optimal settings\n",
    "4. **Deep learning** — `PyTorch` or `TensorFlow` for image/text/sequence data\n",
    "5. **Deployment** — Turn your model into a web app or API\n",
    "\n",
    "---\n",
    "\n",
    "### Thank You!\n",
    "\n",
    "**You can now build, evaluate, and interpret ML models on real biomedical data.**\n",
    "\n",
    "Questions? Ask now!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}