How does Filebeat track file positions and ensure no logs are lost?

Picture a librarian who is simultaneously reading through hundreds of books, switching between them all day. Instead of trying to remember 'I was on page 42 of the blue book on the third shelf,' she keeps a notebook with one line per book, identified by that book's unique barcode rather than its title — because titles get reprinted, covers get replaced, and books get reshelved constantly. Each line records the exact page she reached. If she loses her place and picks the notebook back up, she can find the barcode and resume from that exact page, even if the book's cover looks completely different than it did yesterday. What she cannot survive is someone tearing pages out of a book faster than she can read them — those pages are simply gone before she gets to them.

In Filebeat, that notebook is the registry file, and the barcode is the combination of filesystem inode and device number rather than the file path. This design exists because production log files are rarely stable: logrotate renames access.log to access.log.1, Docker's json-log driver rotates container logs, and applications like payments-api or order-fulfillment truncate and recreate files on deploy. A filename-based bookmark would break constantly under rotation, either re-reading whole files from scratch or losing the tail of a renamed file mid-read. By keying off inode and device, Filebeat treats a renamed file as the same file it was already tracking, and can keep reading from the correct offset uninterrupted.

Internally, each configured input scans its paths and, for every matching file, spins up a harvester — a dedicated goroutine that opens the file and reads new lines as they're appended. Each line becomes an event that flows through libbeat's internal queue toward the configured output (Elasticsearch, Logstash, or Kafka). Crucially, Filebeat does not update the registry offset the moment it reads a line; it waits for an ACK (acknowledgment) from the output confirming the event was durably received. Only then does the registrar component persist the new offset to disk. This ACK-gated commit is what makes the delivery guarantee at-least-once: if Filebeat crashes between reading and receiving the ACK, it re-reads and re-sends those lines on restart rather than assuming they arrived.

At scale — say hundreds of hosts each running Filebeat against payments-api, checkout-worker, and fraud-detection-svc logs — the registry can accumulate thousands of entries, and registry write frequency becomes a real I/O concern, which is why modern Filebeat writes registry state incrementally to a directory rather than rewriting one giant file. Operators should monitor open harvester counts, the gap between events read and events acked, and registry flush latency. A registry that stops advancing while harvesters stay open is a strong signal of a stalled output or a downstream outage, not necessarily lost data — yet.

The gotcha experienced engineers eventually hit is truncation and inode reuse. If a log file is truncated in place (some naive rotation scripts do a hard truncate instead of proper copytruncate coordination) rather than renamed, Filebeat detects the file is now smaller than its recorded offset and resets to zero — but depending on timing, that can produce duplicate reads of whatever was written between truncation and detection. Worse, on filesystems or container runtimes that aggressively reuse inode numbers (fresh volumes after a pod restart, some NFS mounts), a brand-new file can inherit the same inode as a previously fully-processed and pruned file, and Filebeat will assume it's already been shipped — silently skipping real, new log data.