How do you design pipeline templates and shared YAML libraries for reuse across 50+ repositories?
Quick Answer
Create a central template repository with parameterized stage, job, and step templates. Other repos reference templates using resources.repositories with a ref to a specific tag or branch, enabling centralized governance while allowing per-repo customization through parameters.
Detailed Answer
Think of pipeline templates like franchise operations manuals. McDonald's does not let each restaurant invent its own burger-making process — there is a central playbook that defines standard procedures (templates), but each location can customize the menu slightly (parameters) based on local regulations. The corporate team (platform engineering) updates the playbook centrally, and franchises (service teams) adopt changes by updating their reference version.
Azure DevOps YAML templates come in four flavors: step templates (reusable build steps), job templates (complete job definitions), stage templates (full stages with conditions and environments), and variable templates (shared variable definitions). The 'extends' keyword adds a fifth pattern — pipeline templates that constrain what child pipelines can do, enforcing security policies like required scanning steps that cannot be removed.
The central template repository (often called 'pipeline-templates' or 'shared-pipelines') contains versioned templates organized by concern: build templates per language (.NET, Node.js, Python), deploy templates per target (Kubernetes, App Service, Lambda), and security templates (SAST scanning, container scanning, secret detection). Each template accepts parameters that customize behavior — container registry, namespace, environment name, notification channels — while the core logic remains standard.
Consumer repositories reference the template repo using resources.repositories and call templates with the @templateRepo syntax. Version pinning is critical: teams reference a specific tag (refs/tags/v2.1) or branch (refs/heads/stable) rather than main, so template updates do not break consuming pipelines unexpectedly. The platform team releases new template versions through a PR process, tags the release, and teams upgrade by changing their ref.
The production gotcha at scale is template debugging. When a pipeline fails inside a template, the error points to the template file but developers cannot see the expanded YAML easily. Azure DevOps provides a 'Download full YAML' option in the pipeline run that shows the fully resolved pipeline after template expansion. Another challenge is parameter validation — templates should use parameter types (string, boolean, object, stepList) and allowed values to fail fast with clear errors rather than cryptic task failures deep in the pipeline. The extends pattern is essential for security: it prevents pipeline authors from adding arbitrary steps that bypass security scanning.