Monitoring a deployed model: four layers

Shipping a model is the start of its working life. The world changes, upstream data changes, and a model that was accurate in testing can quietly degrade while every server metric stays green. Monitoring an ML system means watching more than uptime. A useful way to organise it is in four layers, from cheap and immediate to slow and definitive.

Layer 1: the system

The same things you would watch for any service.

• Availability and errors: is it up, and how many requests fail?
• Latency: how long predictions take, at typical and worst-case percentiles.
• Resource use: CPU, memory, GPU, queue depth, cost per request.

This layer tells you the service is alive. It says nothing about whether the predictions are any good.

Layer 2: the data going in

Models are sensitive to their inputs. Watch for:

• Schema and validity: missing columns, wrong types, values outside allowed ranges, unexpected nulls.
• Freshness: is the upstream data arriving on time?
• Distribution shift (data drift): do inputs still look like the training data? Compare distributions of key features now versus a reference period, using simple statistics or a drift measure such as population stability index or a two-sample test.
• Pipeline changes: a renamed field or a unit change (dollars to cents) upstream can break a model without a single error.

Input problems are the most common cause of silent failure, and they are detectable before you have any labels.

Layer 3: the predictions coming out

• Distribution of outputs: the share of positive predictions, average score, class balance. A sudden change is a red flag even without ground truth.
• Confidence patterns: many more low-confidence predictions than usual.
• Out-of-range or degenerate outputs: constant predictions, impossible values.
• Segment behaviour: predictions broken down by region, device or customer type, since averages hide local failures.

import numpy as np

def population_stability_index(expected, actual, bins=10):
    edges = np.quantile(expected, np.linspace(0, 1, bins + 1))
    e = np.histogram(expected, edges)[0] / len(expected)
    a = np.histogram(actual, edges)[0] / len(actual)
    e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
    return float(np.sum((a - e) * np.log(a / e)))

Choose alert thresholds from your own history rather than copying a number.

Layer 4: the outcomes

The definitive question: were the predictions right? This needs ground truth, which often arrives late (a loan defaults months later) or never (you only see outcomes for approved cases).

• Delayed labels. Join predictions to outcomes as they arrive and compute accuracy, precision, recall or business metrics over rolling windows.
• Proxy metrics. When labels lag, watch a faster signal that correlates: click-through, complaints, manual overrides.
• Human review samples. Have people label a small random sample regularly.
• Bias in feedback. If the model influences which outcomes you see, your labels are skewed; keep an exploration or random-holdout slice to measure honestly.

Types of drift

• Data drift: input distribution changes (a new customer segment).
• Concept drift: the relationship between inputs and the outcome changes (behaviour shifts after a policy change).
• Label drift: the outcome rate itself changes.

They need different responses: data drift may need re-checking preprocessing; concept drift usually needs retraining.

Alerts that people act on

• Alert on actionable conditions with a named owner and a runbook line ("if input drift on feature X exceeds Y, check the upstream ETL").
• Prefer a few high-signal alerts over dozens of noisy ones.
• Separate page now (serving broken, invalid inputs) from review this week (slow drift).

Close the loop

Monitoring is only useful if it leads somewhere: retraining triggers, rollback to the previous model version, a shadow deployment of a candidate, or a human fallback. Decide in advance what you will do when each layer trips, and keep the last known good model easy to restore.

Common mistakes

1. Watching only server health.
2. No baseline to compare drift against.
3. Waiting for labels that arrive months late, with no earlier signal.
4. Alerts nobody owns.
5. No way to roll back.

A checklist

• Do we log inputs, predictions, model version and timestamps?
• Are input validity and drift monitored?
• Do we track output distributions by segment?
• Is there a path from predictions to outcomes, even delayed?
• Does every alert have an owner and an action?
• Can we roll back quickly?