How do you wire MLflow into CI/CD so that model promotion is as governed as a code deploy?
Quick Answer
Registration is automatic from training pipelines; promotion is a pipeline, not a UI click: a triggered validation job runs quality gates (metrics vs champion, schema checks, bias/latency tests) and only on pass moves the alias — with the registry webhook/event log as the audit trail and alias-repointing as instant rollback.
Detailed Answer
The pattern: (1) training pipelines always log to tracking and register candidate versions with signatures, lineage tags (git SHA, data version, pipeline run ID) — no manual uploads; (2) new-version events (webhooks where available, or polling) trigger a validation pipeline: load the model, score the golden dataset, compare against the current @champion on primary metrics with statistical thresholds, run schema/latency/size checks, and any domain-specific gates (fairness slices, calibration); (3) on pass, the pipeline sets @challenger (or directly @champion for auto-promote flows), tags the version with validation evidence links, and notifies; on fail, it tags the rejection reason; (4) deployment systems resolve aliases at rollout — promoting never rebuilds images, and rollback repoints the alias; (5) humans approve where required via PR-style approvals gating the alias move, not via clicking the registry UI. Everything a human did in a UI click becomes a reviewable, replayable pipeline run. The anti-pattern to call out: 'the registry as shared folder' — models registered manually with no gates, promoted by whoever has access, with Slack as the audit log.
Code Example
# promotion job (triggered on new version)
cand = f'models:/churn/{new_version}'
score = evaluate(mlflow.pyfunc.load_model(cand), golden_df)
champ = evaluate(mlflow.pyfunc.load_model('models:/churn@champion'), golden_df)
if score.auc >= champ.auc - 0.002 and latency_p99_ms(cand) < 40:
client.set_registered_model_alias('churn', 'challenger', new_version)
client.set_model_version_tag('churn', new_version, 'validation_run', run_url)Interview Tip
The phrase 'promotion is a pipeline, not a UI click' plus alias-based rollback covers exactly what interviewers probe for in MLOps governance questions.