How does GitLab CI/CD variable precedence work and what are the different scopes where variables can be defined?

Think of variable precedence like a layered dress code at a company. The CEO sets company-wide rules (instance variables): business casual. The VP of Engineering overrides for the tech department (group variables): jeans are fine. The project manager overrides for a specific team (project variables): t-shirts allowed during crunch. A developer overrides for their task (pipeline/job variables): pajamas during the all-night deploy. The most specific rule wins, but each layer can only override the ones above it.

GitLab CI/CD variables can be defined at six different scopes, each with its own precedence level. From lowest to highest priority: instance-level variables (set by admins, available to all projects), group-level variables (set on a group, available to all projects in the group), project-level variables (set in project Settings > CI/CD > Variables), variables defined in the variables keyword in .gitlab-ci.yml at the global level, variables defined in the variables keyword within a specific job in .gitlab-ci.yml, and variables passed when triggering a pipeline manually or via the API. There are also predefined variables that GitLab automatically sets for every pipeline (like CI_COMMIT_SHA, CI_PIPELINE_ID, CI_JOB_TOKEN), which cannot be overridden. Variables can be marked as protected (only available on protected branches), masked (hidden in job logs), or both.

Internally, when a job is about to execute, the GitLab Runner receives a set of variables from the GitLab server. The server compiles this set by walking through all variable scopes in precedence order, with higher-priority values overriding lower-priority ones. The Runner then injects these variables as environment variables into the job's execution environment. For file-type variables, GitLab writes the variable value to a temporary file and sets the variable to the file path, which is useful for injecting certificates, kubeconfig files, or service account keys. Variable expansion (referencing one variable inside another using $VARIABLE syntax) happens at the Runner level, with GitLab expanding variables in the YAML file during pipeline creation and the Runner expanding them during job execution. This two-phase expansion can cause subtle issues when a variable references another variable defined at a different scope.

In production, a company with multiple environments for inventory-api would leverage variable scoping strategically. Instance-level variables hold organization-wide settings like COMPANY_DOCKER_REGISTRY: registry.acme.com. Group-level variables for the logistics group define SENTRY_DSN and DATADOG_API_KEY shared across all logistics services. Project-level variables store project-specific values: DATABASE_NAME: inventory_db, with protected and masked PRODUCTION_DB_PASSWORD. The .gitlab-ci.yml file defines environment-specific variables within jobs: the staging deploy job sets APP_ENV: staging while the production deploy job sets APP_ENV: production. When a developer triggers a pipeline manually and overrides APP_ENV to debug, the trigger variable takes highest priority and overrides the job-level YAML variable. This layered approach eliminates hardcoded values, centralizes shared configuration, and keeps secrets out of the repository.

A critical gotcha is the masked variable limitation: GitLab can only mask variables whose values are at least 8 characters long and consist of characters from the Base64 alphabet. Short values like true, 3306, or dev cannot be masked, and attempting to mask them will either fail silently or warn in the UI. Another common mistake is expecting group variables to be inherited by subgroups without understanding the full inheritance chain. Variables flow from parent groups to subgroups to projects, but a project-level variable with the same name overrides the group-level one. The most dangerous pitfall is accidentally exposing protected variables by creating a branch that matches a protected branch pattern; audit your protected branch patterns regularly to ensure they only match intended branches.