Tech Study Guide
MLOps Systems
MLOps systems with model registries, feature stores, training pipelines, artifact versioning, reproducibility, batch and online inference, canaries, shadow evaluation, rollback, and cost governance.
MLOps Systems
MLOps is the system around the model: data, training, registry, release, serving, monitoring, incident response, governance, and cost control. The model artifact is only one piece.
System Components
| Component | Job |
|---|---|
| Data catalog | Tracks sources, schemas, ownership, and classification. |
| Feature store | Keeps offline and online features consistent. |
| Training pipeline | Reproducible training and evaluation workflow. |
| Model registry | Stores approved artifacts, metadata, metrics, and lineage. |
| Serving platform | Routes traffic to model/runtime versions. |
| Monitor | Watches data, behavior, latency, safety, and cost. |
| Release controller | Handles canary, rollback, approval, and audit. |
Deployment Modes
| Mode | Use | Risk |
|---|---|---|
| Batch inference | Offline scoring and reports. | Stale predictions and backfill errors. |
| Online inference | Low-latency request/response. | Tail latency and feature freshness. |
| Streaming inference | Continuous event decisions. | Ordering, duplication, exactly-once expectations. |
| Shadow evaluation | Test candidate without user-visible output. | Privacy and extra cost. |
| Canary | Limited production traffic. | Requires per-version monitoring. |
Practical Lab: Model Registry Record
model_id: ticket-escalation-v4
artifact_uri: s3://models/ticket-escalation/v4
training_data: dataset_2026_05_01
feature_schema: features_v7
eval_report: eval_2026_05_02
owner: ml-platform
approved_for: production-canary
rollback_to: ticket-escalation-v3
Cost Governance
Track cost by model, tenant, route, prompt tokens, output tokens, GPU hours, retrieval calls, reranker calls, and human review. Cost surprises are operational incidents.
Study Cards
What does a model registry store besides weights?
Metadata, lineage, metrics, approvals, artifact URI, versions, owners, and deployment status.
Why use shadow evaluation?
It tests a candidate on real traffic without making its output user-visible.
Why is cost governance part of MLOps?
Token, GPU, retrieval, and review costs can regress independently of quality.