How to Prevent Duplicate Cron Job Execution Locks

How to Prevent Duplicate Cron Job Execution Locks

Scheduled background tasks, commonly known as cron jobs, are essential for SaaS workflows (such as sending emails, processing payments, or generating analytics summaries). However, if a cron job is scheduled to run every minute but occasionally takes 90 seconds to complete, a second instance will start while the first is still running.

This concurrency leads to duplicate database entries, processor thrashing, API rate-limit blocks, and race conditions.

If you are asking: How do I prevent the same cron job from running concurrently? This guide covers implementation methods for single-server file locking, distributed Redis locking, database advisory locks, and setting up heartbeat monitors to track silent failures.

1. Single Server locking: File Locks with Flock

If your scheduled task runs on a single Linux server, the native utility flock is the simplest way to prevent overlapping execution. flock manages exclusive locks at the filesystem level.

Add this pattern to your crontab configuration:

* * * * * flock -n /var/run/my_process.lock -c '/usr/bin/php /var/www/app/artisan process:queue'

• -n (non-blocking): If the lock file /var/run/my_process.lock is already held by an active process, flock exits immediately with a return code of 1, skipping the run.
• -c: The command to execute if the lock is successfully acquired.

2. Multi-Server Locking: Redis Distributed Locks

If your SaaS is deployed across multiple virtual machines or Docker containers, a local file lock will not work because each server has its own isolated filesystem.

To manage locks across a cluster, use Redis as a centralized distributed locking mechanism. When the cron task begins, acquire a lock by writing a key with a unique token value and an expiration time:

SET lock:cron:process_queue "token_123" NX EX 300

• NX: Set the key only if it does not already exist. This guarantees mutual exclusion.
• EX 300: Automatically expire the lock after 300 seconds (5 minutes) to prevent deadlock if the process crashes before releasing the lock.

Releasing the Lock Safely

When the job completes, do not use a simple DEL command. If the job took longer than 300 seconds, the lock would have expired, and another process might have acquired it. Deleting the key blindly would release a lock owned by another worker.

Always release the lock atomically using a Lua script that verifies the token value:

if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end

3. Database Advisory Locks

If you are using PostgreSQL or MySQL and do not want to manage a Redis cluster, database advisory locks offer a robust alternative.

In PostgreSQL, acquire a session-level lock using a unique 64-bit integer identifier:

SELECT pg_try_advisory_lock(123456789);

If the query returns true, you have acquired the lock. If it returns false, another server is running the task. The advantage of advisory locks is that they automatically release if the database connection drops.

4. Comparing Cron Locking Strategies

Choose the right locking strategy based on your deployment architecture:

5. Designing for Idempotency

Locks can fail if the job runtime exceeds the lock expiration time:

[\text{Job Execution Time} > \text{Lock Time-To-Live (TTL)}]

If this threshold is crossed, the lock releases prematurely, allowing a duplicate process to run. To prevent this, always design your background jobs to be idempotent (safe to run multiple times with the same input).

Enforce this at the database layer using unique database keys:

CREATE UNIQUE INDEX idx_unique_billing ON customer_charges (invoice_id, billing_cycle);

If a duplicate process attempts to charge a customer twice, the database will throw a unique constraint violation, blocking duplicate charges.

6. Heartbeat Monitoring with Pingzo

A major drawback of locking mechanisms is that they can fail silently. If a script freezes, the lock file or key remains active, blocking all future runs without raising a system alert.

Pingzo prevents this via Cron Heartbeat Monitoring:

• Active Check-ins: Point your cron scripts to ping a Pingzo webhook URL upon completion.
• Stale Job Detection: If Pingzo does not receive a ping at the scheduled interval (e.g., every 5 minutes), it registers a failure.
• WhatsApp Alerting: The on-call engineer receives an immediate alert on WhatsApp to inspect the locked process before your queue backlog stalls.