Docker Compose

Picture an apartment building where every unit gets its own internal intercom system. Inside unit 4B, you can page 'kitchen' or 'front desk' and it just works, because that intercom network only exists within 4B's walls — it has no idea unit 6A also has a room called 'kitchen.' But the building's single street address and its shared elevator are different: only one entity can claim '123 Main St, Unit 1' at a time. Docker Compose networking works exactly like this. Each Compose project is its own private apartment (network), with its own internal name resolution, but published ports are the shared street address — a single, host-wide resource that any project can only claim once.

Compose was designed around per-project networks specifically so that stacks don't need central coordination to avoid name clashes. If every project shared one flat Docker network, then any two teams both naming a service 'redis' or 'api' would break each other's DNS resolution the moment both stacks ran on the same machine — a serious problem for shared CI runners or dev laptops running multiple projects at once. By scoping each project to its own bridge network (named <project>_default unless you override it), Compose lets 'db' mean something different in every project without any conflict.

Internally, when you run docker compose up, Compose first creates (or reuses) a user-defined bridge network for the project. Docker's embedded DNS server, which listens inside that bridge network at 127.0.0.11, registers each container's name and any network aliases as A records the moment the container starts. When checkout-worker's code does a DNS lookup for postgres-db, the container's resolv.conf points it at that embedded DNS server, which resolves the name to postgres-db's current container IP on that same bridge network — and re-resolves it if the container restarts and gets a new IP, which is why you should never hardcode IPs and always connect by service name. This whole mechanism is invisible to anything outside that specific bridge network; a container in a different Compose project's network cannot resolve or route to it at all unless you explicitly join both to an external shared network.

At scale, the failure mode teams actually hit isn't DNS collision (that's already solved by project isolation) — it's port collision. If project A's docker-compose.yml publishes 5432:5432 for its database and project B does the same, whichever starts second gets a hard 'port is already allocated' error, because host port bindings are a single shared namespace across the whole Docker daemon, not scoped per project the way the internal network is. Teams running multiple stacks on one CI host typically fix this by binding to ephemeral or offset host ports (binding host port to 0 lets Docker pick a free one) and only exposing internally.

The gotcha: containers in two different Compose projects cannot talk to each other by service name even if you know both project names, because they're on entirely separate bridge networks by default. Engineers debugging 'why can't service A reach service B' often don't realize the two are actually defined in separate docker-compose.yml files (maybe checked out from two different repos) and were never meant to share a network — the fix isn't a DNS problem to debug, it's adding both services to a common external: true network, and forgetting this is one of the most common multi-repo Compose setup mistakes.

Think of this like troubleshooting why a phone call between two departments in an office isn't connecting. You don't start by suspecting the recipient's phone is broken — you first check whether both extensions are even on the same phone system, then whether dialing the extension number rings anywhere at all, then whether the call connects but nobody picks up, and only last whether the person who answers is refusing the specific request being made. Networking debugging should follow the same outside-in narrowing, because jumping straight to 'the app must be misconfigured' skips several cheaper, faster checks.

Compose intentionally exposes each of these layers separately so you can test them independently: docker network inspect shows you attachment (is the container even plugged into the network), docker exec ... nslookup shows you name resolution, docker exec ... nc -zv shows you raw TCP reachability, and only after those three pass does an actual failed connection point at something inside the application — wrong credentials, wrong database name, or a startup race condition. Skipping straight to reading application logs is the single most common reason engineers waste 30+ minutes on what turns out to be a one-line networks: misconfiguration.

Step by step: first, confirm attachment — run docker inspect checkout-worker and check .NetworkSettings.Networks, and do the same for postgres-db; if they're not on a common network key, that's your root cause immediately, usually because someone added a custom networks: block to one service and forgot the default network no longer auto-attaches once you define an explicit one. Second, exec into checkout-worker and run getent hosts postgres-db — if this fails to resolve, either the network attachment is wrong (step one already caught it) or the service was renamed and stale DNS caching in a long-lived connection pool is still holding an old IP. Third, test the actual port with nc -zv postgres-db 5432 from inside checkout-worker — if DNS resolves but the port doesn't respond, the app inside postgres-db likely isn't listening yet (a health check gap) or is bound to localhost inside its own container instead of 0.0.0.0. Only once TCP connects and you still see errors do you move to application logs for auth failures, wrong database names, or connection pool exhaustion.

At scale, this exact failure pattern is the most common cause of 'flaky' integration tests in CI — the network layer is fine, but the healthcheck is missing or too permissive, so checkout-worker starts before postgres-db's TCP listener is actually open, producing intermittent, hard-to-reproduce failures depending on scheduling luck. Teams that add condition: service_healthy on depends_on convert this from an intermittent CI flake into a deterministic wait, which is why it's worth checking as step zero before diving into DNS at all.

The gotcha: docker compose ps reporting a container as 'healthy' only reflects what its own healthcheck test command checked — it says nothing about whether a *different* service can actually reach it, since healthchecks run from inside the container's own namespace. Engineers regularly get burned assuming 'healthy' means 'reachable from anywhere,' when in reality a container can pass its own loopback-based healthcheck while still being unreachable from a sibling container due to a firewall rule inside the image or a service incorrectly binding to 127.0.0.1 instead of all interfaces.

Running a Compose stack in production is like running a small restaurant out of a single kitchen instead of a chain with a central commissary — if the grill and the fryer both get slammed with orders at the same time, there's no second kitchen down the street to send overflow orders to. Kubernetes has a scheduler that can move a crashed workload to a healthier node; Compose has no such thing. Whatever host is running docker compose up is the entire blast radius, so noticing resource pressure early matters more here than almost anywhere else, because the fallback when something runs out of memory is simply 'the container dies and restarts in place,' not 'gets rescheduled somewhere with more headroom.'

This single-host reality is exactly why Compose stacks need tighter, more proactive monitoring than their Kubernetes equivalents. Compose was built as a local/single-node tool, not a distributed orchestrator, so it deliberately doesn't include the observability and self-healing machinery Kubernetes has — that's the trade-off for its simplicity. Teams running Compose in staging or small production environments need to bolt on their own visibility, because without it, a memory leak in checkout-worker doesn't get flagged and rescheduled elsewhere — it just silently degrades the whole host until postgres-db, which happens to share that host, also starts swapping and slowing to a crawl.

Step by step, the useful signal path looks like this: docker stats (or its scraped equivalent via cAdvisor/Prometheus node exporter) reports live CPU and memory usage per container; compare that against each service's configured mem_limit and cpus in the compose file. As a container's memory usage approaches its limit, the kernel's OOM killer inside that container's cgroup will eventually kill its main process, Docker will report the container exited with code 137, and if restart: always or on-failure is set, Compose immediately restarts it — which, if the underlying leak is still there, restarts, leaks again, and repeats in a crash loop that shows up as a climbing RestartCount in docker inspect. Host-level metrics (available memory, swap usage, disk I/O wait) matter just as much, because containers without explicit limits can consume the entire host's memory and starve every other container, not just themselves.

At production scale, teams scrape docker stats-equivalent metrics into Prometheus via cAdvisor and alert on three tiers: container memory usage above 80% of its limit for 5+ minutes (leading indicator), any container restart in the last 10 minutes (lagging but urgent), and host-level memory pressure or swap activity (systemic risk to every co-located container). Disk is the most commonly forgotten dimension — Postgres write-ahead logs and application log files filling the host's disk will eventually cause every container on that host to fail unpredictably, often with confusing 'no space left on device' errors that look unrelated to the actual disk exhaustion.

The gotcha: a service without an explicit mem_limit doesn't get 'no limit' in a good sense — it gets access to the entire host's memory, meaning one leaking service can starve out completely unrelated, otherwise-healthy containers on the same box, and the OOM killer might pick the *wrong* victim (postgres-db instead of the actual leaking checkout-worker) because Linux's OOM heuristics score based on memory usage and adjustability, not which process actually caused the problem. Engineers who set generous or absent limits on 'just the worker' are often surprised when the database gets killed instead during a memory crunch.

Picture a hotel that reuses the same room number for every guest without checking anyone out first — the moment two guests are assigned room 204 at overlapping times, someone's luggage gets mixed up with someone else's. That's exactly what happens when a CI system runs docker compose up for two concurrent pipeline jobs on the same runner without namespacing them: by default, Compose derives its project name from the current directory name, so two checkouts of the same repo running in parallel both try to create a network and containers named after the same project, and the second job either fails outright or, worse, silently reuses the first job's still-running containers and gets contaminated test data.

Compose's project-name mechanism exists specifically to let multiple independent instances of the same stack run side-by-side without any manual coordination — but it only helps if you actually set a unique name per run instead of relying on the default directory-derived one, which is identical across every checkout of the same repo. CI systems are the textbook case where you want deliberately unique namespacing every single run, because the whole point of CI is parallel, disposable, repeatable execution, and 'repeatable' breaks the instant one run's leftover state can bleed into the next.

Internally, when you pass -p <unique-name> or set COMPOSE_PROJECT_NAME, every resource Compose creates — the bridge network, named volumes, and container names — gets prefixed with that project name instead of the directory name. This means job A's <project-a>_default network and job B's <project-b>_default network are two entirely separate Docker networks even though both were built from the identical docker-compose.yml file, and their containers, despite having the same service names internally (both have a service called 'postgres-db'), get different actual container names like project-a_postgres-db_1 and project-b_postgres-db_1. This is what makes true parallel execution on a shared runner safe.

At CI scale, the two things that still bite teams are host port publishing and leftover volumes. If your compose file hardcodes a fixed host port mapping like 5432:5432, two parallel jobs will still collide on the host's port 5432 regardless of project name, because published ports are a Docker-daemon-wide resource, not scoped per project — the fix is binding the host side to 0 (Docker picks a free ephemeral port) and having your test suite read the actual assigned port via docker compose port postgres-db 5432 rather than assuming 5432. The other recurring issue is teardown: a test failure that causes the pipeline step to exit early without running docker compose down -v leaves orphaned containers and volumes accumulating on the runner until disk fills up, so the teardown must run in a finally-equivalent (a post-step or trap) regardless of whether the tests passed.

The gotcha: docker compose down without -v deletes containers and networks but leaves named volumes behind, so a 'clean' teardown that forgets -v still accumulates orphaned Postgres data volumes run after run, and after enough CI runs the runner's disk fills up in a way that looks like a random, unrelated Docker daemon failure rather than an obvious volume leak — engineers often burn hours looking at the wrong layer before realizing docker system df shows dozens of forgotten anonymous volumes.

Think of it like a base employment contract with addendums. The base contract (docker-compose.yml) spells out your job title, salary band, and reporting line — the things true in every office. A regional addendum (docker-compose.prod.yml) doesn't rewrite the whole contract, it just amends specific clauses: different working hours here, a different bonus structure there. HR doesn't hand you two separate contracts to reconcile yourself; the system merges them into one effective document. Docker Compose's multi-file support works the same way — you keep one canonical description of your stack and layer small, environment-specific patches on top instead of maintaining three near-duplicate 200-line YAML files that drift out of sync.

Compose was designed this way because the alternative — full separate files per environment — is a maintenance trap. If checkout-worker gains a new environment variable in dev, someone has to remember to also add it to staging.yml and prod.yml, and inevitably one gets missed until a production incident reveals it. By making files additive, the base file is the single source of truth for topology (which services exist, how they're networked) while overrides only carry the deltas that are genuinely environment-specific, like credentials, replica counts, or removing a live-reload volume mount that should never exist in production.

Internally, when you run docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d, Compose reads each file in order and performs a deep merge on the resulting data structure. Scalar keys (like image or restart) get overwritten by the later file. List keys like ports and volumes are replaced entirely, not appended — this trips people up because they expect merging to be additive everywhere. Map keys like environment and labels are merged key-by-key. Compose resolves this into a single in-memory Compose model before it ever talks to the Docker Engine API, so what you see with docker compose config (which prints the fully merged, resolved YAML) is exactly what gets sent to create containers, networks, and volumes.

At production scale, teams also lean on profiles to conditionally include services — a debug profile might add a pgAdmin or Redis Commander container that only starts when you pass --profile debug, keeping the production compose file free of debugging tools by default. What to watch: config drift between the base file and overrides (catch this in CI by running docker compose config against each environment combination and diffing against a checked-in snapshot), and secrets accidentally left in an override file that gets committed to git. A common on-call scenario is someone updating docker-compose.yml for a new port mapping but forgetting the prod override still hard-codes the old port, causing a silent mismatch that only surfaces when the load balancer health check starts failing.

The non-obvious gotcha: because list-type keys are replaced wholesale rather than merged, if your base file defines volumes: [./data:/data, ./logs:/logs] and your override only needs to add a third volume, you must repeat all three in the override — Compose will not append your new entry to the base list. Engineers who assume Compose merges lists item-by-item (the way it merges maps) end up silently losing volume mounts in production, and the failure mode looks like a missing directory error deep inside the container rather than an obvious YAML problem.

Imagine a relay race where the baton handoff rule is 'start running as soon as your teammate leaves the starting blocks,' rather than 'start running once your teammate is actually within arm's reach.' The first rule sounds fine until you realize the runner might trip, false-start, or take a few strides before finding their stride — and now you're reaching for a baton that isn't there yet. Docker Compose's default depends_on is that first rule: it guarantees start order, not readiness. Your checkout-worker container gets scheduled to start the instant the postgres container process launches, not once Postgres has actually finished its startup sequence and opened its listening socket for real queries.

Compose was designed this way because 'started' and 'ready' are fundamentally different concepts that only the application itself can answer. Docker doesn't know that Postgres needs to run crash recovery, load extensions, and open a TCP listener before it's actually usable — from Docker's point of view, a container is 'up' the moment its entrypoint process exists. The healthcheck mechanism exists specifically to let each service define, in its own terms, what 'ready' means: for Postgres that's pg_isready succeeding, for an HTTP API that's a 200 response on /health, for a queue consumer that might be a successful connection handshake to the broker.

Internally, when you define a healthcheck block, the Docker Engine runs the specified test command inside the container on a schedule (interval), and gives it timeout seconds to complete. If it fails retries times in a row, the container's health status flips from starting to unhealthy; if it succeeds, it flips to healthy. When a dependent service declares depends_on: db: condition: service_healthy, Compose's orchestration logic polls the dependency's health status before issuing the start call for the dependent container — it literally will not create checkout-worker's container until postgres reports healthy, or it fails the whole up command if postgres never becomes healthy within its healthcheck's retry budget.

At production and CI scale, this matters most during full-stack spin-ups — integration test pipelines that bring up a database, a message broker, and five services are exactly where naive depends_on causes flaky, non-deterministic test failures: the app connects before the DB is ready, throws a connection-refused error, and the whole pipeline run fails intermittently, which teams often misdiagnose as 'flaky tests' rather than a missing health check. What to monitor: healthcheck failure counts in docker inspect (an unhealthy container that never recovers usually means a bad healthcheck test command, not necessarily an unhealthy app), and startup latency creep, since every added health-checked dependency adds to the critical path of docker compose up.

The gotcha experienced engineers hit: healthchecks only run after the container's start_period (a grace window during which failures don't count against retries) has elapsed, and if you don't set start_period for a slow-starting JVM-based service, the healthcheck can exhaust its retries and mark the container unhealthy before the app has even finished its warm-up — causing Compose to declare the dependency permanently broken even though it would have become healthy ten seconds later.

docker-compose.yml declares services, networks, and volumes; docker compose up starts them together with a shared network so they resolve each other by service name. It's great for development and CI, but for production orchestration across many hosts you'd use Kubernetes or Swarm.

Compose creates a network and attaches each service; a service named db is reachable at hostname db on its container port. You don't publish ports for internal traffic — only expose ports to the host with ports: when you need external access. depends_on controls start order (not readiness).

ports: '8080:80' maps host 8080 to container 80, reachable from outside. expose: '80' makes the port available on the Compose network but not the host. For service-to-service traffic you need neither if they share a network — the container port is already reachable by service name.

Bind mounts tie data to a host path; named volumes are managed by Docker and persist across recreation. docker compose down keeps named volumes; add -v to delete them. For databases, always use a named volume so data survives container recreation.

Non-sensitive config goes in environment: or an env_file. The top-level .env feeds variable interpolation like ${TAG}. For secrets, Compose secrets mount files into /run/secrets rather than baking values into the image or env, which are more exposed. Never commit real secrets.

By default up reuses existing images; --build forces a rebuild after code changes. -d frees your terminal and runs services in the background (view logs with docker compose logs -f). Combine as up -d --build for a fresh detached start.

docker compose up --scale web=3 runs three replicas of a stateless service (behind a proxy/load balancer). Compose merges docker-compose.yml with docker-compose.override.yml automatically, or you pass multiple -f files (base + prod) to override images, replicas, and env per environment.