How do cron jobs work in Linux, and what are the best practices for scheduling and managing automated tasks?

Question

Accepted Answer

Cron is the standard Linux job scheduler that runs commands or scripts at specified times and intervals. For SREs, cron jobs handle routine tasks like log rotation, backup verification, certificate renewal checks, metrics collection, and cleanup of temporary files. Understanding cron is essential because poorly managed cron jobs are a frequent source of production issues.

The cron schedule format has five fields: minute (0-59), hour (0-23), day of month (1-31), month (1-12), and day of week (0-7, where both 0 and 7 represent Sunday). An asterisk means "every." For example, 30 2 * * * runs at 2:30 AM daily, 0 */6 * * * runs every 6 hours, and 0 0 * * 0 runs at midnight every Sunday. The @reboot directive runs a command once at system startup.

Edit cron jobs with crontab -e for user-specific jobs or place scripts in /etc/cron.d/, /etc/cron.daily/, /etc/cron.hourly/, etc., for system-level jobs. Use crontab -l to list current jobs. System-level cron files in /etc/cron.d/ have an additional field for the user to run the command as.

Best practices for production cron jobs start with redirecting output. By default, cron emails output to the user, which often goes nowhere. Redirect stdout and stderr to a log file: 30 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1. This creates an audit trail for debugging when jobs fail.

Use flock or a similar mechanism to prevent overlapping executions. If a job takes longer than expected, the next scheduled run might start before the previous one finishes, causing conflicts. The command flock -n /tmp/backup.lock /opt/scripts/backup.sh ensures only one instance runs at a time and skips the run if the lock is held.

Set the PATH and environment explicitly in cron jobs because cron runs with a minimal environment that differs from your interactive shell. Either set PATH at the top of the crontab file or use absolute paths for all commands in your scripts. This is one of the most common reasons cron jobs fail: they work when run manually but fail under cron because a required binary is not in the default PATH.

Monitor cron job execution. A cron job that silently stops running can cause subtle problems. Implement a heartbeat pattern where the job reports success to a monitoring service like Healthchecks.io or a custom endpoint after each successful run. If the heartbeat is missed, an alert fires. This catches both job failures and cases where cron itself stops running.

For complex scheduling needs, consider systemd timers as a modern alternative to cron. Systemd timers offer better logging through journalctl, dependency management, and more flexible scheduling options including randomized delays to prevent thundering herd problems.