Model serving

Serving a model means running it so other systems can send inputs and get predictions.

• Online: answer each request right away (fraud checks at payment).
• Batch: score many records on a schedule (nightly churn scores).
• Ship the preprocessing (like the fitted scaler) with the model.

times = sorted([80, 85, 88, 90, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 105, 110, 115, 120, 250, 300])
print(times[18], "ms p95")

250 ms p95

Remember: Watch p95 latency, not just the average.

Common mistake: Serving the model without the exact preprocessing used in training.