How does Linux memory management work — virtual memory, swap, and OOM killer?

Picture virtual memory as each tenant in an apartment building believing they have the whole building to themselves. Each tenant (process) has their own floor plan (virtual address space), but behind the scenes the building manager (kernel) maps their rooms to actual physical spaces and sometimes stores overflow furniture in a basement locker (swap). If the building fills up completely and someone needs more space, the manager has to evict a tenant. That is the OOM killer.

Virtual memory is the foundation of Linux memory management. Every process gets its own virtual address space, usually spanning 128 TB on 64-bit systems. This space is split into sections: code (text), data, heap (which grows upward via brk/mmap), memory-mapped files, and stack (which grows downward). The kernel keeps page tables that translate virtual addresses to physical page frames, and the hardware MMU (Memory Management Unit) does this translation on every memory access. Pages are normally 4 KB, but huge pages (2 MB or 1 GB) can reduce pressure on the TLB (Translation Lookaside Buffer, which is a small cache of recent address translations) for memory-heavy applications like databases. Linux uses demand paging, which means pages are not actually allocated until you touch them, triggering a page fault that the kernel handles by assigning a physical frame.

Under the hood, the kernel organizes physical memory using the buddy allocator, which manages free pages in power-of-two blocks to reduce fragmentation. On top of that, the slab allocator (SLUB in modern kernels) handles small kernel objects like process structures and inodes efficiently. User-space memory is tracked through Virtual Memory Areas (VMAs) in each process's memory descriptor. The kernel also uses the page cache to keep recently read file data in RAM, which is the main reason a busy server's 'free' memory looks low. Linux aggressively caches file data but will give that memory back when applications need it. The kswapd daemon runs in the background, scanning for inactive pages and moving them to swap when free memory drops below set thresholds.

Swap space extends memory beyond physical RAM by using disk as overflow. When the kernel needs to free up RAM, it writes anonymous pages (heap, stack, anything not backed by a file) to the swap partition or swap file. The vm.swappiness setting (0-200, default 60) controls how aggressively the kernel swaps versus reclaiming file cache. In production, setting swappiness too high on a database server causes horrible latency spikes because disk I/O is thousands of times slower than RAM. Many production setups use vm.swappiness=1 or vm.swappiness=10 and rely on having enough RAM. Kubernetes environments often disable swap entirely (required before Kubernetes 1.22) or use newer swap-aware features. The mkswap and swapon commands manage swap devices, and /proc/swaps and free -h show current usage.

The OOM (Out-of-Memory) killer is the kernel's last resort when both RAM and swap are full. It calculates an oom_score for each process based on how much memory it uses, adjusted by oom_score_adj (a value from -1000 to 1000, where -1000 means 'never kill me'). The process with the highest score gets killed with SIGKILL. A tricky part is Linux's memory overcommit policy (vm.overcommit_memory), which lets processes request more memory than actually exists. The malloc() call succeeds, but if the memory is actually used later, the OOM killer can strike at any random moment, not just during allocation. Production systems should tune oom_score_adj to protect critical services: set -999 for your database and monitoring, and leave application servers at the default. Check dmesg or journalctl for OOM events. Another watch-out is transparent huge pages (THP), which can cause latency spikes during memory compaction. Many database vendors recommend disabling THP entirely.