What is a DAG pipeline in GitLab using the needs keyword, and how does it differ from the default stage-based execution model?

Think of a DAG pipeline like a cooking recipe where dishes are prepared independently. In a traditional kitchen (stage-based), the chef finishes all appetizers before starting any main courses, even if the salad has nothing to do with the soup. In a DAG kitchen, the moment the salad ingredients are prepped, the salad assembly starts immediately while the soup is still simmering. Each dish only waits for its own ingredients, not for every dish in the previous course to be finished.

In GitLab's default execution model, jobs are organized into stages, and all jobs in a stage must complete before any job in the next stage begins. This provides a simple, predictable execution order but creates artificial bottlenecks. For example, if your test stage has a 10-minute integration test and a 1-minute lint check, the deploy stage waits for both to finish even if the deploy job only depends on the build output and lint result. The needs keyword breaks this stage barrier by letting you declare that a job depends on specific jobs rather than an entire stage. When you add needs: [build-frontend] to a job, that job starts as soon as build-frontend completes, regardless of other jobs still running in the build stage. The result is a Directed Acyclic Graph where jobs form a dependency tree rather than a flat sequence of stages.

Internally, GitLab's pipeline processor evaluates the needs declarations to build a dependency graph. When a job completes, the processor checks if any jobs that need it now have all their dependencies satisfied. If so, those jobs are immediately moved from created to pending state and queued for Runner execution. The stage keyword still matters for visual grouping in the pipeline view and as a fallback ordering mechanism, but the actual execution order is determined by the needs graph. GitLab also uses the needs relationship for artifact passing: by default, a job with needs only receives artifacts from the jobs it needs, not from all jobs in previous stages. You can override this with artifacts: true or artifacts: false on each needs entry. There is a configurable limit (default 50) on the number of needs entries per job, enforced to prevent overly complex dependency graphs.

In production, DAG pipelines provide the biggest benefit for monorepo or multi-component projects. Consider a repository for logistics-platform that contains a frontend (React), a backend API (Go), and a worker service (Python). In a stage-based pipeline, the test stage would wait for all three build jobs to finish before any tests start. With needs, the frontend-test job starts the moment frontend-build completes, even while backend-build is still compiling Go code. A real-world pipeline that took 25 minutes with strict stage ordering can drop to 15 minutes using DAG because independent components flow through the pipeline at their own pace. Teams at companies like GitLab itself use needs extensively in their monorepo pipelines to keep CI feedback loops short despite having hundreds of jobs.

A key gotcha is that using needs: [] (an empty array) means the job has no dependencies and starts immediately when the pipeline is created, even before any other job runs. This is useful for jobs like static analysis that do not need any build artifacts. However, forgetting to include a necessary dependency causes the job to run without the required artifacts, leading to confusing failures. Another pitfall is mixing needs with stage-based ordering: if you add needs to some jobs but not others in the same stage, the jobs without needs still follow stage-based ordering while the needs jobs run according to the DAG. This inconsistency can make pipeline behavior hard to predict.