What are Liveness and Readiness probes and why do production Pods need them?

Imagine you manage a call center. A liveness check is like peeking into an agent's booth to see if they're alive and conscious — if they've fainted at their desk (deadlock, infinite loop), you need to wake them up or replace them. A readiness check is different: the agent might be alive and well but currently in training, updating their system, or on a personal call — they're not ready to take customer calls yet. You wouldn't route a customer to them until they signal they're available.

In Kubernetes, the kubelet on each node performs these health checks on every container. A liveness probe answers 'is this container stuck and needs a restart?' If the liveness probe fails (returns non-200 HTTP, exit code non-zero, or TCP connection refused) for the configured number of consecutive failures (failureThreshold), the kubelet kills and restarts the container. This handles scenarios where your application is running but broken — a deadlocked Java thread, an unresponsive event loop in Node.js, or a corrupted internal state.

A readiness probe answers 'can this container handle traffic right now?' If the readiness probe fails, the kubelet tells the Endpoints controller to remove this Pod's IP from all Service endpoints. Traffic stops flowing to this Pod, but it is NOT restarted. This is crucial during startup (your Spring Boot app needs 30 seconds to load), during dependency outages (your cache is temporarily down so you can't serve requests), and during graceful shutdown. When the readiness probe passes again, the Pod is added back to Service endpoints.

There's also a third probe type: startupProbe (added in Kubernetes 1.18). It runs only during container startup and disables the liveness and readiness probes until it succeeds. This is designed for slow-starting applications — a Java app that needs 2 minutes to load. Without a startup probe, you'd have to set the liveness probe's initialDelaySeconds very high, which means Kubernetes would be slow to detect a crash after startup. With a startup probe, you get both: patience during startup and fast failure detection afterward.

The probes support three mechanisms: httpGet (hit an endpoint, check for 200-399 status), exec (run a command inside the container, check for exit code 0), and tcpSocket (try to open a TCP connection to a port). For web applications, httpGet is almost always the right choice. The most dangerous mistake is making a readiness probe check downstream dependencies: if your readiness probe checks 'can I reach the database?' and the database has a brief hiccup, ALL your Pods fail their readiness probes simultaneously, the Service routes to zero Pods, and you have a complete outage — even though your application could have served cached responses or returned a useful error message.