MLOps basics

Model serving

Serving a model means running it so other systems can send inputs and get predictions.

• Online: answer each request right away (fraud checks at payment).
• Batch: score many records on a schedule (nightly churn scores).
• Ship the preprocessing (like the fitted scaler) with the model.

times = sorted([80, 85, 88, 90, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 105, 110, 115, 120, 250, 300])
print(times[18], "ms p95")

250 ms p95

Remember: Watch p95 latency, not just the average.

Common mistake: Serving the model without the exact preprocessing used in training.

APIs for models

A model API takes input as JSON over HTTP, checks it, and returns the prediction as JSON.

• FastAPI or Flask in Python; a POST /predict endpoint.
• Validate inputs: missing fields, wrong types, impossible values.
• Package it in a Docker container so it runs the same everywhere.

import json
x = json.loads('{"hours": 5, "attendance": 0.9}')
print(json.dumps({"predicted_marks": 10 * x["hours"] + 20 * x["attendance"]}))

{"predicted_marks": 68.0}

Remember: Reject bad input with a clear error, never a silent guess.

Common mistake: Trusting every input, so one bad request crashes the server.

Data drift

Data drift is when live inputs start to look different from the training data, so the model's accuracy can quietly drop.

• Compare live feature statistics (mean, spread, categories) with training.
• Concept drift: the link between inputs and the answer itself changes.
• Alert on big shifts, then investigate and retrain if needed.

import numpy as np
train, live = np.array([30, 32, 28]), np.array([48, 52, 50])
print(round(float(live.mean() - train.mean()), 1), bool(abs(live.mean() - train.mean()) > 2 * train.std()))

20.0 True

Remember: Models age. Plan to monitor and retrain.

Common mistake: Deploying once and assuming the model stays accurate forever.

Model monitoring

Model monitoring watches both the system (speed, errors) and the model (inputs, outputs, accuracy) in production.

• Track latency, error rate and request volume.
• Track the mix of predictions; sudden changes often mean a bug.
• Measure accuracy when the real labels arrive, which may be weeks later.

before, after = 0.91, 0.78
print(round(before - after, 2), "drop")

0.13 drop

Remember: A sudden change in predictions is a warning, not good news.

Common mistake: Only watching server uptime while the model's predictions go wrong.

Experiment tracking

Experiment tracking records every training run: settings, data version, metrics and outputs.

• Tools: MLflow, Weights & Biases.
• Fix random seeds so runs can be repeated.
• Pick the best run by validation score, then test once.

runs = [{"id": 1, "lr": 0.1, "val": 0.84}, {"id": 2, "lr": 0.01, "val": 0.89}, {"id": 3, "lr": 0.001, "val": 0.86}]
best = max(runs, key=lambda r: r["val"])
print(best["id"], best["lr"])

2 0.01

Remember: If it isn't logged, you can't reproduce it.

Common mistake: Choosing the run with the best training accuracy.

Model versioning

Versioning models (and their data and settings) tells you which model made each prediction and lets you roll back.

• A model registry tracks versions and stages: staging, production.
• Version the training data too (e.g. with DVC).
• Release carefully: a canary sends a small share of traffic to the new version first.

Example: v6 in production, v7 gets 5% of traffic as a canary. v7's errors rise, so traffic goes back to v6.

Remember: Rolling back should take minutes.

Common mistake: Overwriting the old model file, so there's nothing to roll back to.