Your PyTorch inference service throws 'CUDA out of memory' after running fine for hours. Walk me through how you would debug it.

OOM-after-hours is almost never the model itself — it loaded fine. The usual suspects, in order: (1) missing torch.no_grad(), so every request builds an autograd graph that is retained as long as the output tensor lives; (2) Python references keeping tensors alive — appending raw CUDA tensors to a metrics list or cache instead of calling .item() or .cpu(); (3) fragmentation — allocated memory is fine but there is no contiguous block big enough, common with highly variable input shapes; (4) unbounded inputs — one client sends a 10x-longer sequence and peak memory spikes. Diagnose with torch.cuda.memory_allocated() vs memory_reserved() over time (steady climb = leak; flat allocated but OOM = fragmentation), torch.cuda.memory_summary(), and the memory snapshot profiler. Fixes map one-to-one: inference_mode, .item()/.detach().cpu() on anything you keep, expandable_segments:True or bucketed padding for fragmentation, and hard caps on batch/sequence size at the API layer.