How do you use Variable Groups, Key Vault integration, and pipeline secrets securely in Azure DevOps?

Think of secrets management in Azure DevOps like a hotel safe system. The hotel (Key Vault) stores all valuables in a central vault with audit logging, access policies, and rotation schedules. Guests (pipelines) access their valuables through a controlled reception desk (Variable Group linked to Key Vault) using their room key (service connection). They never carry the vault combination themselves — they just get temporary access when they need it.

Variable Groups are collections of key-value pairs defined at the project or library level. They can be referenced by multiple pipelines, creating a single source of truth for shared configuration like environment URLs, feature flags, or connection strings. Variable Groups have their own RBAC — you control which pipelines can access them and which users can edit them.

The Key Vault integration transforms Variable Groups from static storage into a dynamic secret proxy. When you link a Variable Group to an Azure Key Vault, the group does not store secret values — it references them. At pipeline runtime, Azure DevOps fetches the current secret values directly from Key Vault. This means secret rotation in Key Vault is automatically picked up by the next pipeline run without any pipeline changes. You select which Key Vault secrets to map into the Variable Group, and they appear as pipeline variables.

Pipeline-level secrets (variables marked as 'secret' in YAML or the UI) have special behavior: they are masked in all log output (replaced with ***), cannot be passed to scripts in forked repository PR builds (preventing secret exfiltration), and cannot be read as output variables without explicit opt-in. The isSecret flag on a variable means Azure DevOps actively prevents accidental exposure.

The critical gotcha is that Variable Groups linked to Key Vault require the pipeline's service connection to have GET permission on Key Vault secrets. If you use a different service connection for deployment than for Key Vault access, you need to grant permissions carefully. Another common issue is that Key Vault-linked variables do not expand in compile-time expressions (like ${{ variables.x }}) — they are only available at runtime ($(x)) because they are fetched during execution. This breaks template conditions that try to use secrets for branching logic.