Redis high availability – Master & slave

Master & slave is what we call master-slave replication, that is, after the host data is updated, it is automatically synchronized to the standby machine according to the configuration and policy. The master is mainly written, and the slave is mainly read.

Master & slave has two main functions:

Redis replication core mechanism

Master slave replication process

Main process:

Incremental replication subprocess:

If the master slave network connection is lost during full replication, incremental replication will be triggered when salve reconnects to the master.

The master directly obtains some lost data from its own backlog and sends it to slave node. The default backlog is 1MB; The master obtains data from the backlog according to the offset in PSYNC sent by slave.

Breakpoint continuation

Since redis 2.8, breakpoint retransmission of master-slave replication has been supported. If the network connection is broken during the master-slave replication, you can continue to copy where you copied last time, instead of copying a copy from scratch.

Master node will create a backlog in memory. Both master and slave will save a replica offset and a master ID. the offset is saved in the backlog. If the network connection between the master and the slave is broken, the slave will make the master continue to replicate from the last replica offset; If the corresponding offset is not found, a full copy is performed.

Expired key processing

The slave will not expire the key, but will only wait for the master to expire the key. If the master has expired a key or eliminated a key through the LRU, a del command will be simulated and sent to the slave.

Other concepts

Heartbeat: the master and slave nodes send heartbeat information to each other. By default, the master sends heartbeat every 10 seconds, and the save node sends heartbeat every 1 second. Every time the master receives a write command, it writes data internally and sends it asynchronously to the slave node

Offset: the master keeps accumulating offsets on itself, and the slave keeps accumulating offsets on itself. The slave will report its own offset to the master every second, and the master will also save the offset of each slave. The main reason is that both the master and slave need to know the offset of their respective data in order to know the data inconsistency between them

Backlog: the master node has a backlog, which is 1MB by default; When the master node copies data to the slave node, it will also synchronously write a copy of the data in the backlog; It is mainly used for incremental replication after full replication is interrupted.

Master run ID: Info server can see the master run ID. If the master node is located according to the host + IP, it is unreliable. If the master node is restarted or the data changes, the slave node should be distinguished according to different run IDS, and full replication should be performed for different run IDS; If you need to restart redis without changing the run ID, you can use the redis cli debug reload command

PSYNC: the slave node uses PSYNC to copy from the master node. PSYNC runid offset master node will return response information according to its own situation. Fullresync runid offset may trigger full replication, or continue may trigger incremental replication