How do you use pipeline stages, jobs, and steps — and what is the difference between each?

Think of this hierarchy like organizing a construction project. Stages are the major project phases: Foundation, Framing, Electrical, Finishing. Jobs within a stage are the crews that can work simultaneously: within the Framing stage, the north-wall crew and south-wall crew work in parallel on different agents. Steps are the individual actions each crew member performs sequentially: measure, cut, nail, inspect. A crew member cannot start nailing before cutting, but two crews can frame different walls at the same time.

Stages represent the highest-level divisions in a pipeline and map to logical phases of your delivery workflow. Common stage patterns include Build, Test, Deploy-Staging, Deploy-Production. Stages execute sequentially by default but can run in parallel when configured with appropriate dependsOn relationships. Each stage boundary is a checkpoint: if a stage fails, subsequent dependent stages are skipped. Stages also serve as the attachment point for environment approvals — when a deployment job within a stage references an environment with approval checks, the entire stage pauses until approval is granted. In the Azure DevOps UI, stages appear as distinct columns in the pipeline run visualization, giving teams a clear view of how far a change has progressed.

Jobs are the unit of agent allocation. Each job runs on exactly one agent — either a Microsoft-hosted agent from a pool or a specific self-hosted agent matching demanded capabilities. Multiple jobs within a stage can run in parallel on different agents, which is how you achieve parallelism for independent work like running unit tests on Linux while simultaneously running them on Windows. Jobs have their own workspace: files created in one job are not automatically available in another job, even within the same stage. To share data between jobs, you must publish and download pipeline artifacts or use output variables. Deployment jobs are a special job type that track deployment history and support deployment strategies.

Steps execute sequentially within a job on a single agent. Steps can be built-in tasks (task: DotNetCoreCLI@2), scripts (script: npm test), or template references. Every step runs in the same workspace with access to files checked out or created by previous steps in the same job. Steps can set output variables that subsequent steps reference, enabling data flow within a job. Each step has optional conditions, timeout settings, and continue-on-error flags. The checkout step at the beginning of a job clones the repository, and subsequent steps operate on that checked-out code.

The execution model has important implications for resource usage and cost. Since each job occupies an agent for its entire duration, minimizing job count reduces parallel agent requirements. However, splitting work into multiple jobs enables parallelism and isolation: if a code analysis job fails, it does not block the unit test job from completing. The tradeoff is agent allocation overhead — each job requires agent provisioning time (30-90 seconds for Microsoft-hosted agents). For short tasks, the overhead of spinning up a new agent can exceed the task execution time, making a single job more efficient.

The production gotcha is misunderstanding workspace isolation between jobs. A common mistake is building code in Job A and expecting the build artifacts to exist in Job B without explicitly publishing them. Each job starts with a fresh workspace (for Microsoft-hosted agents) or a cleaned workspace (for self-hosted agents by default). Teams must use PublishPipelineArtifact and DownloadPipelineArtifact tasks to pass files between jobs, or use output variables for simple string values. Another issue is agent pool exhaustion: if you define 10 parallel jobs but your self-hosted pool only has 3 agents, 7 jobs queue and wait, serializing execution and potentially causing timeouts.