Skip to Content

Regression Metrics (MSE, MAE, RMSE, R-squared, MAPE)

When to use

Forecast demand, price, energy or any real-valued outcome — measure with the right loss function for the right cost.

Analogy

Being a chef. MAE is typical error in grams; MSE heavily penalises burning the food; R-squared says how much of the variation you explained.

Data-flow diagram

   MAE  = mean(|y - y_hat|)        easy to interpret in raw units
   MSE  = mean((y - y_hat)**2)      heavy penalty on large errors
   RMSE = sqrt(MSE)                 same units as y
   R^2  = 1 - SS_res/SS_tot         variance explained
   MAPE = mean(|y - y_hat| / |y|)  percentage (bad near 0)

Deep explanation

MAE is robust to outliers, easy to interpret. MSE/RMSE penalise large errors disproportionately. R-squared is variance explained — high values can hide systematic errors. MAPE is percentage error — breaks near zero. For business reporting use MAE or RMSE in the business unit; for optimisation MSE has a clean gradient.

Examples

Example 1

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np
y = np.array([3.0, 5.0, 7.5, 9.0, 11.0])
p = np.array([3.2, 4.7, 8.0, 8.5, 11.4])
print('MAE:', mean_absolute_error(y, p))
print('RMSE:', mean_squared_error(y, p, squared=False))
print('R^2:', r2_score(y, p))

mean_squared_error(y, p, squared=False) is RMSE in the original units of y.

Example 2

def mape(y, p):
    y, p = np.array(y), np.array(p)
    return np.mean(np.abs((y - p) / np.where(y==0, 1, y))) * 100
print('MAPE:', mape(y, p), '%')

MAPE behaves badly near zero — guard with np.where or accept INF.

Example 3

def pinball(y, p, q):
    diff = np.array(y) - np.array(p)
    return np.mean(np.maximum(q * diff, (q - 1) * diff))
print('pinball@0.5:', pinball(y, p, 0.5))
print('pinball@0.9:', pinball(y, p, 0.9))

Pinball loss is the canonical quantile-regression loss; useful for inventory and risk modelling.

Common mistake

Treating R-squared as the headline — it can be high while predictions are systematically wrong. Also reporting MAPE on data with zeros — the metric blows up.

Key takeaway

MAE for robust default; RMSE when outliers are costly; MAPE only when relative percent matters AND no zeros.

Production Failure Playbook

Failure scenario 1: rmse-pretends-outliers-dont-exist

Failure scenario 2: mape-near-zero-dominator