How does LLMOps differ from classical MLOps — what carries over and what breaks?
Quick Answer
The lifecycle discipline carries over (versioning, eval gates, staged rollout, monitoring), but the artifacts change: you rarely train the model, so 'training pipeline' becomes prompt/RAG/config engineering; evaluation shifts from labeled metrics to golden sets with LLM-judge scoring; monitoring adds token cost, safety, and hallucination; and the 'model registry' becomes prompts, retrieval configs, and provider/model versions.
Detailed Answer
What transfers directly: treat every behavior-changing artifact as versioned and gated — except now those artifacts are prompts, chat templates, retrieval parameters, tool definitions, and the provider model ID (which vendors update under you; pin dated snapshots where offered). Evaluation changes character: classical ML has ground-truth labels and crisp metrics; LLM outputs need golden sets scored by rubric-driven LLM judges calibrated against periodic human review, plus hard checks where possible (schema validation, citation presence, safety filters). Monitoring adds new axes: token spend per request/tenant (a cost regression is a production incident), time-to-first-token, refusal/fallback rates, and quality sampling on live traffic. New failure modes with no classical analog: prompt injection, context-window truncation, provider-side model updates changing behavior with zero deploys on your side, and non-determinism making 'reproduce the bug' probabilistic. What breaks: drift detection on input distributions means little when inputs are free text — you monitor output quality and task success instead. The punchline: the governance instinct is identical, the artifact list and eval machinery are new.
Code Example
# the versioned unit in LLMOps
release:
prompt: support-answer@v23
model: claude-sonnet-5@2026-05-snapshot
retriever: {index: kb_v7, k: 8, reranker: ce-v2}
gates: [schema_valid>0.995, judge_score>0.87, cost_per_req<\$0.012]Interview Tip
The crisp contrast: 'in MLOps the weights change and the prompt doesn't exist; in LLMOps the weights are frozen and everything around them is the deployable'. Then name provider-side model updates as the drift source teams forget.