Redis high availability - master-slave replication, sentinel mode and cluster mode
summary
As an in-memory database, Redis has become a core component of many Internet applications with its extremely high performance and rich functions. However, although stand-alone Redis has excellent performance, it has problems such as single point of failure and poor scalability, which limits its application in high availability and high scalability.
To improve the usability and scalability of Redis, Redis provides a variety of architectural modes: master-slave replication, sentinel mode, and cluster mode. Each mode plays an important role in different scenarios.
1. Master-Slave Replication
1. Overview
Redis's master-slave replication (Master-Slave Replication) is one of the most basic high-availability architectures.
In this architecture, Redis deploys one master node and multiple slave nodes.
The master node is responsible for handling all write operations (such as SET, DEL, etc.), while the slave node backs up data and provides read operation services by synchronizing the master node's data.
2. Working principle
- Master node(Master) is the source node of the data, and all write operations are first executed on the master node.
- Slave node(Slave) Ensure data consistency by replicating the data of the master node. The slave node periodically obtains snapshots of data (RDB) or incremental data (AOF) from the master node for synchronization.
- Synchronization methodThe synchronization between the slave node and the master node is divided into full synchronization and incremental synchronization. During the first connection, full synchronization is performed to completely copy the master node's data to the slave node; subsequent incremental synchronization only synchronizes the change data on the master node.
3. Pros and cons
advantage：
- Read and write separation: Master-slave replication can allocate read requests to slave nodes, thereby reducing the pressure on the master node and improving the concurrency capability of the system.
- Data redundancy: The slave node provides data backup, enhancing the fault tolerance of the system.
shortcoming：
- Single point of failure: After the master node fails, the system will face data unavailability. Although data can be recovered by the slave node, it is necessary to manually promote the slave node to the master node.
- Synchronization delay: There may be delays in master-slave synchronization, especially when write operations are frequent, data synchronization of slave nodes may be lagging, resulting in inconsistent read requests.
4. Applicable scenarios
Master-slave replication is suitable for scenarios where read operations are much larger than write operations, such as cache systems, log systems, etc.
In these scenarios, the slave node can be used as a read replica, reducing the pressure on the master node and improving read performance.
5. Configuration steps
5.1 Configure the master node
In Redis, the master node default configuration can work as a master node, so starting the master node is very simple.
# Start the master noderedis-server /etc/redis/
5.2 Configuring the slave node
The configuration of slave nodes is a little more complicated. The main thing is to tell Redis that it is a slave node and specify the master node information to be replicated.
Add the following to the slave node's configuration file:
# Configure the slave node and specify the master nodeslaveof 192.168.1.1 6379
Here, 192.168.1.1 is the IP address of the master node, and 6379 is the port of the master node. Through this line configuration, the slave node will automatically establish a connection with the master node and start synchronizing the data.
After configuring the slave node, start the slave node:
# Start the slave noderedis-server /etc/redis/
5.3 Verify master-slave replication
Execute the INFO replication command on the master node to view the replication status:
127.0.0.1:6379> INFO replication #Output:# role:master # connected_slaves:1 # slave0:ip=192.168.1.2,port=6379,state=online,offset=12345,lag=0
Execute the same command on the slave node to view the status of the slave node:
127.0.0.1:6380> INFO replication #Output:# role:slave # master_host:192.168.1.1 # master_port:6379 # master_link_status:up
2. Sentinel mode
1. Overview
To compensate for the single point of failure of master-slave replication, Redis introduced Sentinel mode. Redis Sentinel is a highly available solution that ensures high availability of Redis clusters through automatic failover and master-slave monitoring.
In sentinel mode, in addition to the master and slave nodes, multiple sentinel nodes (Sentinels) are also required. The main responsibilities of the Sentinel Node include:
- Monitors the health status of master and slave nodes.
- When the master node goes down, failover will be automatically performed to promote a slave node to the master node.
- After failure recovery, update the master-slave relationship and reconfigure the slave node.
- Provides service discovery function, and the client can obtain the current master node information through sentries.
2. Working principle
- monitor: The sentinel node periodically sends PING requests to the master and slave nodes to monitor the health status of the node.
- Failover: When multiple sentinel nodes confirm that the master node is unavailable, the sentinel will automatically elect a slave node and promote it to a new master node. After that, the new master node will start receiving write operations, and other slave nodes will start synchronizing the new master node data.
- Configuration update: The sentinel node will broadcast new master node information to all slave nodes, and the slave node will automatically synchronize data to the new master node.
3. Pros and cons
advantage：
- Automatic failover: When a master node fails, the sentry will automatically detect and promote a slave node to a master node, without manual intervention, ensuring high availability of the system.
- High availability: Multiple sentinel nodes provide high reliability and avoid the impact of a single sentinel failure.
- Service Discovery: Sentinel can dynamically provide the current master node information to the client, ensuring that the client always accesses the latest master node.
shortcoming：
- High complexity: Sentinel mode is more complex than simple master-slave replication, and is more difficult to configure and operate.
- Long recovery time: Although Sentinels can fail over automatically, in the event of a failure, the failover process will still have a certain delay, resulting in the system being unavailable in a short period of time.
4. Applicable scenarios
The Sentinel mode is suitable for scenarios with high requirements for high availability, such as real-time business systems, online services, etc.
It can ensure that the Redis cluster can still provide stable services when the master node fails.
5. Configuration steps
5.1 Configure sentinel file
Sentinel configuration is usually performed in files. In the file, you need to specify the master node to be monitored, set monitoring parameters, etc.
Here is a simple configuration file example:
# port 26379 sentinel monitor mymaster 192.168.1.1 6379 2 sentinel down-after-milliseconds mymaster 5000 sentinel failover-timeout mymaster 60000 sentinel parallel-syncs mymaster 1
- sentinel monitor mymaster <master_ip> <master_port>: Monitor the master node named mymaster, with an IP of 192.168.1.1 and a port of 6379. At least 2 sentries are required to confirm the master node failure.
- sentinel down-after-milliseconds mymaster 5000: The master node is judged to be a fault if it does not respond within 5000 milliseconds.
- sentinel failover-timeout mymaster 60000: The timeout time for failover is 60 seconds.
Start the sentinel
When starting Sentinel, execute the following command:
redis-sentinel /etc/redis/
5.2 Verify Sentinel Mode
The status of the sentry can be viewed through the following command:
127.0.0.1:26379> INFO sentinel #Output:# sentinel_masters # name=mymaster # ip=192.168.1.1 # port=6379 # quorum=2 # status=ok
3. Cluster mode (Cluster)
1. Overview
For the needLarge-scale distributed deploymentandHorizontal expansionRedis providesRedis Cluster. Unlike master-slave replication and sentinel modes, Redis clusters use sharding to distribute data to multiple nodes, thus supporting higher data storage capabilities and stronger scalability.
In the Redis cluster, data will be allocated in the form of hash slots. The Redis cluster has a total of 16384 hash slots, and each node in the cluster is responsible for a portion of the hash slots. Data allocation and management are automatically completed, and the client can directly connect to any node in the cluster, and the cluster will route the request to the correct node based on the requested data hash value.
2. Working principle
- Sharding: Redis Cluster divides all key-value pairs into 16384 hash slots according to the hash algorithm, and allocates these hash slots to multiple nodes. Each node stores a portion of the data and is responsible for the keys within the hash slot range.
- Data replication: Each master node has one or more slave nodes for data backup, and the slave node can take over if the master node fails.
- Failover: When the master node fails, the cluster will automatically promote the slave node of the master node to a new master node to ensure data availability.
3. Pros and cons
advantage：
- High scalability: The cluster mode supports horizontal scaling, and can dynamically increase or decrease nodes as needed, and easily process massive data.
- Automatic failover: The Redis cluster provides an automatic failover mechanism to ensure that the system can automatically recover when the node is down.
- Distributed Storage: Data is stored on different nodes in shards, which can support super-large data storage.
shortcoming：
- High complexity: Compared with master-slave replication and sentinel modes, the configuration and operation and maintenance of Redis clusters are more complex, and requires a deeper understanding of sharding, data migration, cluster management, etc.
- Poor cross-node operation performance: If the client's operation involves multiple hash slots, the Redis cluster needs to communicate across nodes, and the performance will be degraded.
4. Applicable scenarios
The cluster model is suitable for scenarios that require large-scale data storage and high concurrency, such as social media, e-commerce, advertising delivery and other scenarios.
Redis clusters are the most suitable architectural pattern for data storage and fast read and write operations that require support for billions of keys.
5. Configuration steps
5.1 Configure cluster nodes
In cluster mode, cluster-related parameters need to be enabled in the configuration file of each Redis node:
# cluster-enabled yes cluster-config-file cluster-node-timeout 5000
- cluster-enabled yes: Enable cluster mode.
- cluster-config-file: Specify the cluster configuration file to save the cluster's node information.
- cluster-node-timeout 5000: Set the node timeout time.
5.2 Start the cluster node
Start multiple Redis instances, each of which needs to use the above configuration file. Suppose we have 6 nodes (3 master nodes, 3 slave nodes):
# Start each noderedis-server /etc/redis/
5.3 Creating a cluster
Use the redis-cli tool to create a cluster and assign slaves to each master node:
# Create a clusterredis-cli --cluster create <node1>:6379 <node2>:6379 <node3>:6379 <node4>:6379 <node5>:6379 <node6>:6379 --cluster-replicas 1
This command creates a cluster with 3 master nodes and 3 slave nodes.
5.4 Verify cluster status
Verify the cluster status by:
redis-cli -c -h <node_ip> -p 6379 cluster info
Summarize
In Redis's high availability architecture,Master-slave copy、Sentinel modeandCluster modeEach has different applicable scenarios and advantages. Choosing the appropriate architectural model needs to be determined based on business needs, system size, data volume and failure recovery time requirements.
- Master-slave copyIt is suitable for scenarios with more read operations and fewer write operations, and can effectively realize read and write separation and improve system performance.
- Sentinel modeThrough automatic failover and monitoring, the high availability of Redis system is guaranteed and is suitable for business scenarios with high high availability requirements.
- Cluster modeProvides the ability to scale horizontally and is suitable for large-scale data storage and high-concurrent request scenarios.
The above is personal experience. I hope you can give you a reference and I hope you can support me more.