Java – enables existing spring batch applications to run on multiple nodes

We have an existing spring batch application, and we hope it can run on multiple nodes

The scalability documentation for spring batch involves code changes and configuration changes

I just want to know if this can be achieved through configuration changes (it's good to add a new class and connect it in the configuration, but I just want to avoid code changes to existing classes)

Thank you very much for your help

Solution

It really depends on your situation Specifically, why run on multiple nodes? What is the bottleneck you are trying to overcome? The two typical out of the box solutions for spring batch to expand on multiple nodes are remote blocking and remote partitioning Both are master / slave configurations, but each has a different use case

When the processor in step is the bottleneck, remote blocking is used In this case, the master node reads the input and sends it to the remote node for processing through the spring integration channel After processing the item, the result will be returned to the main program for writing In this case, reads and writes are done locally Although this helps parallelize processing, it requires I / O hits because each item is sent over the line (and transmission needs to be guaranteed, such as ala JMS)

Remote partitioning is another case In this case, the master device generates a description of the input to be processed for each slave device and transmits the description only over the line For example, if you are processing records in the database, the master server can send a series of row IDS (1-100101-200, etc.) to each slave server Reads and writes occur locally to slaves and do not require guaranteed delivery (although useful in some cases)

Depending on your usage, both options can be completed with minimal (or no) new classes There are several different places to find information about these features:

>Spring batch integration GitHub repository – spring batch integration is a project that supports the above use cases You can read more here: https://github.com/spring-projects/spring-batch-admin/tree/master/spring-batch-integration >My remote partitioning example – this discussion takes place through remote partitioning and provides a working example running on cloud foundry (currently only applicable to CF V1, but the update of CF2 will be released in a few days) The configuration is almost the same, except that rabbit's connection pool is different: https://github.com/mminella/Spring-Batch-Talk-2.0 A video of this presentation can be found on YouTube: http://www.youtube.com/watch?v=CYTj5YT7CZU >Gunnar hillert's speech on spring batch and spring integration: This is presented on springone2gx 2013, including many examples: https://github.com/ghillert/spring-batch-integration-sample

In any of these cases, remote partitioning should be possible through zero new classes Remote partitioning usually requires you to implement a new class (partitioning program)