An effective method to synchronize two tables in different servers using java

We have a source table updated from various external systems I require the target table (on a different server) to synchronize with this source table The target table is not an exact copy of the source table. Some data processing must be completed before inserting / updating data into the target table

I thought of the following logic

Every 15 minutes, we run the Java consumer code, which gets the record of the creation timestamp instead of the previously updated record, and stores it in the cachedrowset The stored procedure is called with cachedrowset as a parameter, where the data processing is completed and the data is inserted / updated into the target table

Do you believe that the above mentioned is an effective way because we process more than one million records every time we update?

When deleting records in the source table, they will not be copied in the above method! What can you suggest to do in this situation

Solution

It is similar to the technology of database for savepoint and rollback As long as there are some changes in the source table, such as crud Retain the change script according to the desired format of the target table You can periodically push these changes to the target server Since the source table is updated by various external systems, you need to set triggers on the source table to keep the script log