How does caching work in GitLab CI/CD and what strategies can you use to optimize pipeline speed with cache configuration?

Think of GitLab CI caching like a workbench in a woodworking shop. Each morning (new pipeline), instead of going to the hardware store to buy fresh screws, nails, and sandpaper, you check your workbench drawer (cache) first. If you find what you need, you save the trip and start building immediately. If the project requirements changed (new lock file), you know the drawer contents are stale, so you make the trip, buy fresh supplies, and refill the drawer for tomorrow. The key to the drawer is labeled with the project plan version (cache key), so different plans get different drawers.

Caching in GitLab CI/CD is configured using the cache keyword within a job. The paths property specifies which directories to cache, commonly node_modules, .pip, vendor/bundle, or .m2/repository. The key property determines how the cache is identified and when it is considered valid. If two jobs have the same cache key, they share the same cache archive. The simplest strategy is a fixed key like default, which means every pipeline reuses the same cache. A better strategy is $CI_COMMIT_REF_SLUG, which creates a per-branch cache so that feature branches do not pollute each other's caches. The most sophisticated strategy uses cache:key:files, which generates the key from the hash of specified files (like package-lock.json or Gemfile.lock), ensuring the cache is invalidated exactly when dependencies change.

Internally, caching works differently from artifacts. While artifacts are stored on the GitLab server and passed between stages within a pipeline, caches are stored on the Runner's local filesystem or in a configured S3-compatible backend. When a job starts, the Runner checks if a cache archive exists for the specified key. If found, it downloads and extracts the archive into the working directory before running the script. After the job completes (regardless of success or failure), the Runner creates a new archive of the cached paths and uploads it. The policy keyword controls this behavior: pull means only download the cache (never upload), push means only upload (never download), and pull-push (the default) does both. Using pull-only on test jobs that consume but never change dependencies avoids unnecessary upload time. The fallback_keys feature allows specifying alternative cache keys to try if the primary key misses, enabling a graduated freshness strategy.

In production, a team working on logistics-dashboard with a Node.js frontend and Python backend would configure layered caching. The frontend jobs use cache:key:files pointing to package-lock.json, caching node_modules. When a developer updates dependencies (changing the lock file), the cache key changes automatically, forcing a fresh npm ci. Meanwhile, all other pipelines where the lock file has not changed hit the cache and skip the download entirely, saving 30-90 seconds per job. The backend jobs cache the pip download directory keyed to requirements.txt. For further optimization, the team splits the cache into a preparation job with policy: push that runs npm ci once, followed by multiple test jobs with policy: pull that only consume the cache. This prevents three test jobs from redundantly uploading the same cache archive.

A critical gotcha is the difference between cache and artifacts. Caches are best-effort and may not be available, so your job must work even if the cache is empty (always run npm ci, not npm install). Artifacts are guaranteed and used for passing build outputs between stages. Another common mistake is caching the wrong directory; for example, caching node_modules works but caching the npm cache directory (~/.npm) is often more reliable because npm ci deletes node_modules before installing. Also, be aware that caches are scoped to the Runner: if you have five Runners, a cache created on Runner 1 is not available on Runner 2 unless you use a distributed cache backend like S3.