How do you capture and analyze a thread dump to diagnose a hung Java application, and why does a single dump often mislead you?
Quick Answer
Capture at least two or three thread dumps a few seconds apart with jstack or jcmd, since a single snapshot can't distinguish a thread that's briefly waiting from one that's permanently stuck — only comparing multiple dumps reveals which threads are stuck in the exact same place over time. Look for BLOCKED threads waiting on the same lock across dumps, and let jstack's built-in deadlock detector flag any true circular lock dependencies automatically.
Detailed Answer
Diagnosing a hung application from one thread dump is like judging whether traffic on a highway is truly gridlocked from a single photograph — a photo can't tell you if that car has been sitting still for twenty minutes or just happened to be stopped for a split second when the shutter clicked. You need at least two photographs a few seconds apart to see whether the same cars are still in the exact same spots, which is the only real evidence of a genuine jam rather than normal, momentary traffic flow. A Java thread dump is exactly that snapshot — it captures the state of every thread in the JVM at one instant, and a thread showing as WAITING or BLOCKED in a single dump might resolve itself half a second later as part of completely normal operation.
This is why the standard practice is to always take multiple dumps in quick succession — typically three, ten seconds apart — rather than relying on one. jstack (or its modern equivalent, jcmd <pid> Thread.print) was designed to be a cheap, safe, repeatable operation specifically so engineers could do this comparison without meaningfully perturbing the running application; unlike a heap dump, which can pause the JVM for a noticeable stop-the-world moment on a large heap, a thread dump is comparatively lightweight.
Step by step: capture dump one, wait ten seconds, capture dump two, wait another ten seconds, capture dump three. Then look at each thread's state across all three: RUNNABLE means the thread is actively executing (or waiting on native I/O, which JVM thread states can't distinguish from real CPU work) — a thread pegged in RUNNABLE across all three dumps at the same stack frame is a hot spin loop or expensive computation, not a hang. BLOCKED means the thread is waiting to acquire a Java monitor (a lock) that another thread currently holds — if the same threads show BLOCKED on the same lock across all three dumps, that's a real, sustained contention problem. WAITING or TIMED_WAITING usually means a thread is idle in a pool waiting for work, which is completely normal and not evidence of a hang by itself. Critically, jstack automatically runs deadlock detection as part of every dump and will print an explicit 'Found one Java-level deadlock' section with the full cycle of threads and locks involved, which is the single most valuable piece of automatically-derived evidence you can get.
At production scale, the most common real-world pattern behind a 'hung' service isn't a classic deadlock at all — it's every HTTP-handling thread ending up BLOCKED waiting on a shared resource like a database connection pool, because the pool itself is exhausted (all connections checked out, none returned, due to a slow or leaking downstream call). This looks exactly like a hang from the outside (the app stops responding to new requests) but the actual root cause is upstream capacity, not a code-level deadlock, and the thread dump reveals this clearly once you notice dozens of threads all BLOCKED at the exact same stack frame inside the connection pool's acquire method.
The gotcha experienced engineers hit: a thread reported as RUNNABLE isn't necessarily doing useful CPU work — the JVM's thread state model can't distinguish 'genuinely computing' from 'blocked in a native OS call like a socket read,' so a thread stuck waiting on a slow network call from a misbehaving downstream service will show as RUNNABLE, not WAITING, in the dump. Engineers who only search dumps for BLOCKED or WAITING states and ignore RUNNABLE threads that never move across multiple dumps miss exactly this class of hang, which is common when a downstream service silently stops responding without closing the TCP connection.