Implementation of distributed timed task framework based on Spring + quartz

Problem background

Our company is a fast-growing start-up company with 200 people. Its main business is related to tourism and hotels. The application iteration update cycle is relatively fast. Therefore, developers spend more time to keep up with the pace of iteration and lack control over the whole system

How to implement the company's scheduled tasks before clustering

In the initial stage, the access volume of the application is not so large. One server is fully used, and there are many scheduled tasks to be executed in the application

With the cluster, the company implements the scheduled tasks

With the increase of users, the number of accesses increases. One server can not meet the requirements of high concurrency. Therefore, the company deploys applications to the cluster, and the front end is through nginx proxy (the application server IP may be isolated by firewall, avoiding the direct access by using IP + port + application name).

In the cluster environment, the same scheduled tasks will be executed by each machine in the cluster. In this way, the scheduled tasks will be executed repeatedly, which will not only increase the burden on the server, but also cause additional unexpected errors due to the repeated execution of scheduled tasks. Therefore, the company's solution is: according to the number of clusters, To evenly distribute the tasks in the scheduled tasks to each machine in the cluster (the average score here means that the previous scheduled task was originally run on one machine. First, artificially divide the task into several parts and let all machines execute it)

At present, there are some defects in the implementation of timed tasks in clusters

At present, the company's method of processing scheduled tasks in the cluster is not a real distributed processing method, but a pseudo distributed (commonly known as the native method within the company). An obvious defect of this method is that when the machine in the cluster goes down, the whole scheduled tasks will hang up or cannot be completed at one time, which will have a serious impact on the business

Solutions to defects (the focus of this article)

Using spring + quartz to build a set of real distributed timing task system, through consulting relevant materials, we know that quartz framework supports distributed timing tasks natively

Development ide: IntelliJ idea

JDK version: 1.8

Spring version: 4.2 six

Quartz version: 2.2 one

Spring and quartz integrated configuration

Quartz properties file

Related class description

The autowiringspringbeanjobfactory class is designed to use spring annotations in the scheduler. If you do not use annotations, you can directly use springbeanjobfactory instead of this class

Test results:

Since there is only one computer, I have opened ports 8080 and 8888 to test. I set the above timing task to run every 10 seconds.

When I only start port 8080, I can see that the console prints a statement every 10 seconds

In the comparison test of two ports started at the same time, it can be seen that only one port is running the timing task

After the port that is running the scheduled task is closed, the previous port that is not running starts to take over and continue to run the scheduled task

So far, we can clearly see that in distributed scheduled tasks (or clusters), only one scheduled task will run at the same time.

Entire demo address: http://xiazai.jb51.net/201701/yuanma/spring-cluster-quartz_jb51.rar

The above is the whole content of this article. I hope it will be helpful to your study, and I hope you can support programming tips.