How do you optimize Git for extremely large repositories, including filesystem monitor integration, multi-pack indexes, and the maintenance scheduler?

Think of optimizing Git for large repos like tuning a search engine for a massive database. The naive approach scans every record for every query (git status stats every file). Fsmonitor is like adding a change log: instead of scanning the entire database, you check what changed since the last query. Multi-pack-index is like adding a unified search index across multiple data partitions. Bitmap indexes are like pre-computed lookup tables for common ancestry queries. And the maintenance scheduler is like running database vacuum and index rebuilds during off-peak hours.

The filesystem monitor (fsmonitor) integration addresses the most visible bottleneck: git status. By default, Git stats every file in the working directory to detect modifications, which becomes unbearably slow with hundreds of thousands of files. Enabling core.fsmonitor connects Git to a filesystem watcher (Facebook's Watchman or Git's built-in fsmonitor--daemon) that tracks which files have actually changed since the last Git operation. When git status runs, it queries the monitor for the list of changed files and only stats those, reducing status time from minutes to milliseconds. The built-in fsmonitor--daemon (available since Git 2.36) runs as a background process per repository and uses platform-specific APIs (FSEvents on macOS, ReadDirectoryChangesW on Windows, inotify on Linux) for change tracking.

The multi-pack-index (MIDX) and bitmap indexes address the packfile lookup and network transfer bottlenecks. In repositories with many packfiles (common in frequently-pushed repos or those avoiding full repacks), each git object lookup requires searching multiple .idx files. MIDX creates a single unified index across all packfiles, enabling O(1) lookups regardless of pack count. Bitmap indexes (.bitmap files) pre-compute reachability information: for selected commits, they store a bitmap of all objects reachable from that commit. This makes git fetch and git push dramatically faster because the server can instantly compute which objects the client needs without walking the entire commit graph. The pack.writeBitmaps and repack.writeBitmaps config options control bitmap generation during repacking.

Git maintenance (git maintenance start) automates the entire optimization lifecycle. It schedules periodic background tasks using platform-specific schedulers (cron on Linux/macOS, Task Scheduler on Windows): prefetch fetches from remotes hourly without updating branches (reducing fetch latency during actual pulls), commit-graph rebuilds the commit-graph daily with incremental updates, gc runs periodic garbage collection, and incremental-repack manages packfile consolidation. The geometric repacking strategy (--geometric=2) ensures packfiles follow a size progression: each pack is at least 2x the next smaller one, bounding the number of packs to O(log N) while avoiding expensive full repacks. For server-side repositories, the midx and pack-refs tasks keep index structures current. The maintenance register command adds multiple repositories to the scheduler without restarting it.

A critical gotcha: fsmonitor has platform-specific quirks. On macOS, FSEvents occasionally misses events during heavy I/O, causing stale results. On Linux, inotify has a per-user watch limit (default 8192 on many distros) that large repos can exhaust; increase it with sysctl fs.inotify.max_user_watches=524288. On NFS and other network filesystems, fsmonitor is unreliable because change notifications do not propagate across the network. Another trap: bitmap indexes consume additional disk space and require recomputation during repacking. For repositories with extremely frequent pushes, the bitmap maintenance overhead may exceed the performance gains. Also, the maintenance scheduler creates persistent background processes and cron jobs that persist beyond the terminal session. Use git maintenance unregister to clean up when removing a repository. Finally, the untracked cache (core.untrackedcache = true) complements fsmonitor by caching the list of untracked files and their directory modification times, avoiding redundant directory scans.