Your checkout-worker JVM service shows p99 latency spiking to 4 seconds every 90 seconds with no visible CPU spike in top — how do you determine whether this is GC-related and which log or tool confirms it?

This kind of clockwork-regular slowdown is like a toll booth that, once every ninety cars, suddenly closes every lane for four seconds to count the cash drawer before reopening — from a distance the traffic flow looks normal almost all the time, but anyone stuck at exactly that moment experiences a hard stop, and it happens on a predictable rhythm rather than randomly. A stop-the-world GC pause behaves exactly like this: for the small fraction of a second (or, in a bad case, seconds) that a Full GC runs, every application thread is frozen so the collector can safely walk and compact memory, and then everything resumes as if nothing happened — until the next time the Old generation fills up again.

The reason CPU monitoring often misses this is a sampling and averaging problem: top typically refreshes every 1-3 seconds and shows an average utilization over that window, so a GC pause that fully saturates all cores for 500ms can get smoothed into an unremarkable-looking average if the sampling window doesn't line up precisely, especially on a multi-core box where the GC threads are a small fraction of total cores. This is exactly why 'no CPU spike' and 'GC pause' are not contradictory — the pause is real and CPU-intensive, it's just short and thread-count-limited enough to hide inside typical monitoring granularity.

Step by step to confirm: first, enable (or check if already enabled) unified JVM GC logging with -Xlog:gc*, which timestamps every GC event including its type (Minor vs Full) and duration; if a Full GC event's timestamp lines up with the observed 4-second latency spike, that's near-certain confirmation. Second, if you can't restart the process to add logging, use jstat -gcutil <pid> 1000 to stream live generation occupancy and GC counts once per second — watching the Old generation (O column) climb steadily and then drop sharply, paired with the FGC (full GC count) column incrementing at that exact moment, confirms the same thing without needing a log file. Third, correlate the 90-second period itself: a suspiciously regular interval like this usually means a fixed-size cache, a scheduled batch job, or a metrics-flushing routine that allocates a large, similarly-sized batch of objects on a fixed timer, filling Old generation to its trigger threshold on a near-identical cadence every time.

At production scale, once GC is confirmed as the cause, the fix path branches: if pause duration is the core problem, moving from a throughput-oriented collector to a low-pause collector like G1 (or ZGC/Shenandoah for very large heaps needing sub-millisecond pauses) directly targets pause time instead of total throughput. If the *root cause* is the 90-second periodic allocation spike itself, the better fix is addressing that pattern directly — batching smaller, or spreading a bulk cache refresh across a window instead of doing it all in one allocation burst — since a differently-tuned GC still has to deal with the same fundamental memory churn, just with a different pause profile.

The gotcha: engineers who see 'no CPU spike' and rule out GC based on that alone will spend hours chasing network flakiness, downstream latency, or application code before ever checking the GC log — when the actual fix would have taken two minutes if GC logging had been on by default. The deeper lesson experienced engineers learn the hard way is that GC logging should be enabled unconditionally in every production JVM service from day one, because it's nearly free to leave running and it's the single fastest way to rule GC in or out during exactly this kind of investigation, instead of needing a restart mid-incident just to turn logging on.