Ensuring Singleton Execution in Distributed Systems with ShedLock and Redis

Introduction

In modern software architecture, applications are often deployed across multiple instances for scalability and reliability. While this distributed approach brings numerous benefits, it also introduces challenges, particularly when dealing with scheduled tasks. How do you ensure that a scheduled job runs only once across all instances? Enter ShedLock, a simple yet powerful library designed to solve this exact problem.

In this article, we’ll delve into the intricacies of ShedLock, explore how it prevents concurrent execution of scheduled tasks in distributed environments, and provide hands-on examples using Spring Boot with Redis as the lock provider.

The Problem: Concurrent Scheduled Tasks in Distributed Systems

Consider an application deployed on multiple servers or instances. If this application has a scheduled task (like a cron job), each instance will execute the task independently at the scheduled time. This can lead to:

• Duplicate Processing: The same data or operation is processed multiple times.
• Data Inconsistency: Conflicting operations may corrupt the data.
• Resource Contention: Multiple instances compete for the same resources, leading to performance degradation.

Example Scenario:

An e-commerce platform sends daily promotional emails to users. If the email-sending task is scheduled without any synchronization mechanism, each application instance will send emails, resulting in users receiving multiple copies.

Introducing ShedLock

ShedLock is an open-source library that ensures only one instance of a scheduled task executes at any given time in a distributed environment. It achieves this by leveraging a shared external store (like Redis) to coordinate locks between instances.

Key Features:

• Framework Agnostic: Works with various scheduling frameworks, including Spring’s @Scheduled.
• Multiple Lock Providers: Supports relational databases, Redis, MongoDB, ZooKeeper, and more.
• Non-Intrusive: Minimal changes required to existing code.
• Configurable: Allows fine-tuning of lock duration, names, and behaviors.

How ShedLock Works

1. Lock Acquisition: Before executing a scheduled task, an instance tries to acquire a lock in a shared store.
2. Lock Verification: If the lock is obtained, the instance proceeds with the task. If not, it skips execution.
3. Lock Release: After the task completes, the lock is released or expires based on configuration.

This mechanism ensures that at most one instance runs the task at any given time.

Need help with Spring Framework? Master Spring TER, a ChatGPT model, offers real-time troubleshooting, problem-solving, and up-to-date Spring Boot info. Click master-spring-ter for free expert support!

Setting Up the Environment

We’ll demonstrate ShedLock using a Spring Boot application with Redis as the lock provider.

Prerequisites

• Java Development Kit (JDK) 11 or higher
• Maven or Gradle build tool
• Redis server
• IDE (IntelliJ IDEA, Eclipse, etc.)

Creating a Spring Boot Project

You can create a new Spring Boot project using Spring Initializr or your preferred method.

Dependencies:

• Spring Web
• Spring Data Redis
• Spring Boot Starter Scheduling
• ShedLock Core
• ShedLock Provider for Redis

Maven pom.xml Dependencies:

<dependencies>
    <!-- Spring Boot Starter Web -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <!-- Spring Boot Starter Scheduling -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>
    <!-- Spring Data Redis -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <!-- ShedLock Core -->
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-core</artifactId>
        <version>5.5.0</version>
    </dependency>
    <!-- ShedLock Provider for Redis -->
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-provider-redis-spring</artifactId>
        <version>5.5.0</version>
    </dependency>
</dependencies>

Configuring Redis

Ensure your Redis server is running and accessible. Update your application.properties or application.yml with the Redis connection details.

Example application.properties:

spring.redis.host=localhost
spring.redis.port=6379

If your Redis server requires authentication, include:

spring.redis.password=your_redis_password

Setting Up ShedLock

1. Enable Scheduling

Enable scheduling in your Spring Boot application by adding @EnableScheduling to your main application class.

@SpringBootApplication
@EnableScheduling
public class ShedLockRedisDemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(ShedLockRedisDemoApplication.class, args);
    }
}

2. Configure ShedLock with Redis

Create a configuration class to set up ShedLock with the Redis lock provider.

@Configuration
public class ShedLockConfig {
    @Bean
    public LockProvider lockProvider(RedisConnectionFactory connectionFactory) {
        return new RedisLockProvider(connectionFactory);
    }
}

This configuration tells ShedLock to use Redis as the lock provider, utilizing Spring Data Redis’s RedisConnectionFactory.

Implementing a Scheduled Task with ShedLock

Let’s create a scheduled task that runs every minute.

@Component
public class MyScheduledTask {
    private static final Logger logger = LoggerFactory.getLogger(MyScheduledTask.class);
    
    @Scheduled(cron = "0 * * * * *") // Every minute
    @SchedulerLock(name = "MyScheduledTask_scheduledTask", lockAtMostFor = "PT1M")
    public void scheduledTask() {
        logger.info("Executing scheduled task");
        // Task implementation
    }
}

Explanation:

• @Scheduled: Defines the schedule (every minute in this case).
• @SchedulerLock: Provided by ShedLock, ensures the lock is acquired before execution.
• name: Unique identifier for the lock.
• lockAtMostFor: Maximum duration to hold the lock. This prevents deadlocks in case the task fails to release the lock.

Testing the Application

Running Multiple Instances

To simulate a distributed environment, run multiple instances of your application. You can do this by:

• Running the application in your IDE multiple times.
• Building the application into a jar and running it in separate terminal windows:

java -jar target/shedlock-redis-demo-0.0.1-SNAPSHOT.jar --server.port=8081
java -jar target/shedlock-redis-demo-0.0.1-SNAPSHOT.jar --server.port=8082

Observing the Logs

With ShedLock in place, you’ll notice that only one instance logs the execution of the scheduled task each minute.

Instance 1 Log:

INFO  c.e.s.MyScheduledTask - Executing scheduled task

Instance 2 Log:

(No log output for the task execution)

Understanding Lock Parameters

• lockAtMostFor: Maximum time the lock should be held. This is a safety net to prevent deadlocks.
• lockAtLeastFor: Minimum time the lock should be held. Ensures the task doesn’t run again too soon.

Example with lockAtLeastFor:

@SchedulerLock(
    name = "MyScheduledTask_scheduledTask",
    lockAtMostFor = "PT1M",
    lockAtLeastFor = "PT30S"
)
public void scheduledTask() {
    // Task implementation
}

In this configuration:

• The lock will be held for at least 30 seconds.
• If the task completes quickly, the lock remains until 30 seconds have passed.
• Prevents tasks from running too frequently.

Best Practices

• Unique Lock Names: Ensure each scheduled task has a unique lock name.
• Configure Appropriate Lock Durations: Set lockAtMostFor slightly longer than the maximum expected task duration.
• Handle Exceptions: Implement proper exception handling to avoid unintended lock releases.
• Monitoring: Keep an eye on Redis keys to monitor lock statuses.
• Redis Configuration: Ensure Redis persistence is appropriately configured if you need locks to survive restarts.

Advanced Configurations

Using Annotations at the Class Level

You can define default lock configurations at the class level.

@Component
@SchedulerLock(name = "MyScheduledTask_scheduledTask", lockAtMostFor = "PT1M")
public class MyScheduledTask {
    @Scheduled(cron = "0 * * * * *")
    public void scheduledTask() {
        // Task implementation
    }
}

Customizing Redis Key Prefix

By default, ShedLock uses the prefix shedlock: for Redis keys. You can customize this if needed.

@Bean
public LockProvider lockProvider(RedisConnectionFactory connectionFactory) {
    return new RedisLockProvider(connectionFactory, "custom-prefix:");
}

Now, the keys will have the prefix custom-prefix:.

Using Redis Cluster or Sentinel

If you’re using Redis Cluster or Sentinel, ensure your RedisConnectionFactory is appropriately configured.

Example for Redis Sentinel:

@Bean
public RedisConnectionFactory redisConnectionFactory() {
    RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
        .master("mymaster")
        .sentinel("127.0.0.1", 26379);
    return new LettuceConnectionFactory(sentinelConfig);
}

Handling Failures and Recovery

Handling Exceptions in Scheduled Tasks

Ensure that your scheduled tasks handle exceptions properly to prevent locks from being held indefinitely.

@Scheduled(cron = "0 * * * * *")
@SchedulerLock(name = "MyScheduledTask_scheduledTask", lockAtMostFor = "PT1M")
public void scheduledTask() {
    try {
        // Task implementation
    } catch (Exception e) {
        logger.error("An error occurred during scheduled task execution", e);
        // Optionally rethrow or handle accordingly
    }
}

Dealing with Redis Unavailability

If Redis becomes unavailable, ShedLock will not be able to acquire or release locks. It’s crucial to ensure high availability for your Redis instance.

Conclusion

ShedLock provides an elegant and straightforward solution to a common problem in distributed systems: preventing concurrent execution of scheduled tasks. By integrating ShedLock into your Spring Boot application with Redis as the lock provider, you can ensure that your scheduled tasks run reliably and without conflict, regardless of how many instances are deployed.

Key Takeaways:

• ShedLock uses a shared store (Redis) to coordinate task execution.
• Minimal configuration changes are required to integrate ShedLock with Redis.
• Supports various storage providers for flexibility.
• Helps maintain data integrity and application stability.

References

• ShedLock GitHub Repository
• ShedLock Documentation
• Spring Scheduling Documentation
• Spring Data Redis Documentation

Thank you for reading! If you found this article helpful, feel free to share it with your peers or leave a comment below.