What's the difference between static and dynamic Jenkins agents, and what production tradeoff does each involve?
Quick Answer
Static agents are persistent machines/VMs registered with Jenkins permanently, always available but idle (and costing money/resources) when no builds are running. Dynamic agents are provisioned on-demand (e.g. via a Kubernetes plugin spinning up a pod, or a cloud plugin launching an EC2 instance) for the duration of a single build and destroyed afterward, matching capacity to actual demand but adding provisioning latency to the start of every build and requiring the agent image/template itself to be properly maintained.
Detailed Answer
A static agent is registered once (via a fixed IP/hostname or a long-running agent process) and simply sits available in Jenkins's agent pool whether or not it's currently running a build — simple to reason about and has zero cold-start latency for a build to begin, but represents wasted capacity (and cost, for cloud-hosted agents) during quiet periods, and configuration drift on long-lived agent machines is a real maintenance burden over time (tool version mismatches, leftover state from prior builds).
Dynamic agents are provisioned just-in-time per build — the Kubernetes plugin, for example, launches a fresh pod (from a defined pod template/container image) when a build is queued and needs an agent, runs the build inside it, and tears the pod down afterward. This scales cleanly with actual build volume (no idle capacity cost) and gives every build a clean, reproducible environment with no leftover state from a previous build — but it adds pod-scheduling and image-pull latency to the start of every single build, and if the underlying cluster/cloud provider itself is under strain, agent provisioning can become a bottleneck exactly when build demand is highest.
Most production setups at any real scale use dynamic agents for the bulk of workload-variable CI (matching the scaling story) while sometimes keeping a small static pool for latency-sensitive or highly specialized workloads (e.g. a build requiring an expensive one-time license activation that's cheaper to keep warm than to reprovision constantly).
Interview Tip
Bring up cold-start latency as the concrete cost of dynamic agents — many candidates only mention 'scaling' as the benefit without acknowledging the corresponding tradeoff.