Skip to Content

AUC: ROC vs PR (Threshold-Free Scoring)

When to use

Score a binary classifier for separation quality without choosing a threshold.

Analogy

ROC asks how well the model separates two clouds of points; PR asks how well it surfaces rare positives in a haystack of negatives.

Data-flow diagram

   ROC: TPR vs FPR across thresholds -> AUC-ROC
   PR:  Precision vs Recall across thresholds -> AUC-PR
   imbalanced data   -> PR-AUC is more honest
   balanced data     -> ROC-AUC is fine

Deep explanation

AUC-ROC plots TPR against FPR across all thresholds. AUC-PR plots Precision against Recall. AUC-ROC is encouraging even on imbalanced data because the high negative count keeps FPR artificially low; PR-AUC fixes that. Pick PR-AUC whenever the positive class is rare. AUC is THRESHOLD-INDEPENDENT.

Examples

Example 1

from sklearn.metrics import roc_auc_score, average_precision_score
y_true = [0, 0, 1, 1, 0, 1, 0, 1, 0, 1]
y_score = [0.1,0.4,0.35,0.8,0.2,0.9,0.3,0.7,0.25,0.65]
print('ROC-AUC:', roc_auc_score(y_true, y_score))
print('PR-AUC:', average_precision_score(y_true, y_score))

roc_auc_score: area under ROC — classic balanced metric.

Example 2

from sklearn.metrics import roc_curve
import numpy as np
fpr, tpr, thr = roc_curve(y_true, y_score)
idx = np.argmin(np.abs(tpr - 0.9))
print(f"at recall 0.9 -> threshold {thr[idx]:.2f}, FPR {fpr[idx]:.2f}")

average_precision_score: area under PR — safer for imbalanced data.

Example 3

import matplotlib.pyplot as plt
from sklearn.metrics import RocCurveDisplay, PrecisionRecallDisplay
RocCurveDisplay.from_predictions(y_true, y_score); plt.show()
PrecisionRecallDisplay.from_predictions(y_true, y_score); plt.show()

RocCurveDisplay and PrecisionRecallDisplay are the human-friendly visualisers.

Common mistake

Reading AUC-ROC as the final answer on imbalanced data. A 0.95 ROC-AUC on a 1 percent positive dataset can still be useless.

Key takeaway

ROC-AUC for balanced problems; PR-AUC for rare-class. AUC measures ranking ability, NOT calibration, NOT business value.

Production Failure Playbook

Failure scenario 1: auc-fooled-by-imbalance

Failure scenario 2: auc-ignored-calibration