How does the cron daemon decide when to run a job, and why do overlapping cron jobs silently stack up and cause outages?
Quick Answer
The cron daemon, crond, wakes up once every minute, reads all configured crontabs, and for each entry checks whether the current minute, hour, day, month, and weekday match that entry's schedule fields; if it matches, it forks a new process to run the command, completely independent of whether a previous invocation of that same job is still running. Because cron has no concept of 'is the last run still in progress,' a job that occasionally takes longer than its scheduling interval gets a second, then third, then more overlapping instance launched on top of it, and without explicit locking those overlapping instances can compete for the same resources and cascade into an outage.
Detailed Answer
Picture a train station where a train departs exactly on the printed schedule every 10 minutes, no matter what, even if the previous train hasn't actually finished unloading passengers on the platform yet. The schedule board doesn't check whether the last train finished its business; it only checks whether 10 minutes have passed since the last scheduled departure time. Normally this is fine because trains reliably finish in under 10 minutes. But the one day a train breaks down and takes 25 minutes to clear the platform, the next scheduled train arrives anyway, and now there are two trains trying to use the same platform at once, a collision the schedule itself never anticipated or prevented in any way.
Cron was built in the earliest days of Unix specifically as a simple, minute-resolution time-based trigger, a design intentionally kept dumb and stateless, checking only wall-clock time against a schedule, with zero awareness of whether a previously-launched job is still executing. This simplicity is exactly why cron has remained essentially unchanged and universally reliable for decades: there's very little to go wrong in the scheduling logic itself. But that same simplicity means cron offers no built-in protection against the previous-run-is-still-going scenario, unlike more modern schedulers that track job state explicitly as part of their design.
Internally, crond wakes up every 60 seconds, reads the parsed crontab entries, from per-user crontabs typically stored under /var/spool/cron/, the system-wide /etc/crontab, and drop-in files under /etc/cron.d/, and for each entry evaluates whether the current minute, hour, day-of-month, month, and day-of-week fields match. Any entry that matches gets forked as a brand-new child process running under the environment and permissions of whichever user owns that crontab. Critically, cron doesn't check a PID file, doesn't ask whether a previous instance of this command is already running, and doesn't wait; it just forks and moves on to evaluate the next matching entry in sequence.
In production, this becomes dangerous for jobs whose runtime is close to or sometimes exceeds their schedule interval. For example, a billing-reconciler job scheduled every 5 minutes that normally finishes in 2 minutes but occasionally takes 8 minutes under heavy load. During that one slow run, cron launches a second instance at the next 5-minute mark anyway, and now two instances are both reading and writing the same reconciliation state, which can produce duplicate charges, double-counted records, or lock contention against a shared database table that cascades into broader slowness for other unrelated jobs and queries hitting the same tables.
The gotcha most engineers miss until they've been burned by it: adding a simple pgrep-based check inside the job script feels like a fix but has a subtle race condition. Between the pgrep check and actually starting the real work, a nearly-simultaneous second cron-triggered invocation can pass the same check before the first instance's process name shows up in the process table, so both proceed anyway. The robust fix is flock, a proper kernel-level file lock, wrapping the job's execution, which atomically ensures only one instance can hold the lock at a time, with the second invocation either exiting immediately or waiting, rather than trusting a process-name check that has an inherent timing gap built into it.