current position:Home>Redis cluster cluster installation

Redis cluster cluster installation

2022-01-27 01:45:39 Sun batian

Redis-cluster colony

Concept

Redis-Cluster Use a no center structure , Each node holds the data and the entire cluster state , Each node is connected to all other nodes .

redis-cluster

Structural features

  1. be-all redis Nodes interconnect with each other (PING-PONG Mechanism ), Internally, binary protocols are used to optimize transmission speed and bandwidth .
  2. Node fail It only takes effect when more than half of the nodes in the cluster are detected to have failed .
  3. The client and redis Node is directly connected , You don't need the middle proxy layer . The client does not need to connect to all the nodes of the cluster , Connect to any of the available nodes in the cluster .
  4. redis-cluster Map all the physical nodes to [0-16383]slot On ( It doesn't have to be equal ),cluster Responsible for maintaining node<->slot<->value.
  5. Redis The cluster is prearranged 16384 A barrel , When need is in Redis Place one in the cluster key-value when , according to CRC16(key) mod 16384 Value , Decided to put a key In which bucket .

cluster Node allocation

Now we are three master nodes, which are :A, B, C Three nodes , They can be three ports on a machine , It can also be three different servers . that , Using the Hashimoto trough (hash slot) How to allocate 16384 individual slot Words , The three nodes are responsible for slot The interval is :

  • node A Cover 0-5460;

  • node B Cover 5461-10922;

  • node C Cover 10923-16383.

get data

If you store a value , according to redis cluster Hash slot algorithm : CRC16(‘key’)384 = 6782. Then you'll take this key Is allocated to B Yes . Again , When I connect (A,B,C) Any node wants to get ’key’ This key when , It's the same algorithm , Then jump inside to B Get data on the node

Above reference

How to vote

The voting process is all in the cluster master Participate in , If more than half master Node and current master Node communication exceeds cluster-node-timeout Time set , Think of the current master Node down .

If one of the nodes master Hang up , its slave Will replace the previous master The location of the has become a new master, After the previous recovery, it will become slave.

How to determine whether a node is unavailable

  1. If the cluster is arbitrary master Hang up , And the current master No, slave( copy ). Cluster into fail state , It can also be understood as a cluster slot mapping [0-16383] Enter when incomplete fail state .
  2. If more than half of the clusters master Hang up , Whether or not slave, Cluster into fail state , When the cluster is not available , All operations on the cluster are not available , received ((error) CLUSTERDOWN The cluster is down) error .

summary

A simple understanding of ,redis-cluster Cluster is the combination of master-slave mode and Sentinel Integration and implementation of patterns .

Cluster building

Environmental preparation

host ip port
192.168.1.251 7001,7002
192.168.1.252 7001,7002
192.168.1.253 7001,7002

install redis

Download from official website redis Of tar package , And unzip the installation

#  decompression 
tar -zxvf redis-5.0.14.tar.gz
#  Enter folder 
cd redis-5.0.14
#  compile 
make
#  install 
make install

Test startup

redis-server redis.conf

image-20211216172334132

Modify the configuration file

mkdir /usr/local/redis-5.0.14/cluster

cp /usr/local/redis-5.0.14/redis.conf /usr/local/redis-5.0.14/cluster/redis_7001.conf

cp /usr/local/redis-5.0.14/redis.conf /usr/local/redis-5.0.14/cluster/redis_7002.conf

chown -R 777 /usr/local/redis-5.0.14

mkdir -p /data1/redis/cluster/{
    redis_7001,redis_7002} && chown -R 777 /data1/redis

redis_7001.conf

# vim /usr/local/redis-5.0.14/cluster/redis_7001.conf

bind 192.168.1.251
port 7001
daemonize yes
pidfile "/var/run/redis_7001.pid"
logfile "/usr/local/redis-5.0.14/cluster/redis_7001.log"
dir "/data1/redis/cluster/redis_7001"
masterauth 123456
requirepass 123456
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7001.conf
cluster-node-timeout 15000

redis_7002.conf

# vim /usr/local/redis-5.0.14/cluster/redis_7002.conf

bind 192.168.1.251
port 7002
daemonize yes
pidfile "/var/run/redis_7002.pid"
logfile "/usr/local/redis-5.0.14/cluster/redis_7002.log"
dir "/data1/redis/cluster/redis_7002"
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7002.conf
cluster-node-timeout 15000

The configuration of other machines is consistent

start-up redis service

redis-server /usr/local/redis-5.0.14/cluster/redis_7001.conf

tail -f /usr/local/redis-5.0.14/cluster/redis_7001.log

redis-server /usr/local/redis-5.0.14/cluster/redis_7002.conf

tail -f /usr/local/redis-5.0.14/cluster/redis_7002.log

Create clusters

If redis The version is relatively low , You need to install ruby. Choose one machine to install ruby that will do

stay redis-5 in redis-trib.rb The functions of are integrated into redis-cli in , Greatly simplified redis Cluster deployment of , The speed of group deployment is accelerated , It is also convenient for later maintenance and capacity expansion .

redis-cli -a 123456 --cluster create 192.168.1.251:7001 192.168.1.251:7002 192.168.1.252:7001 192.168.1.252:7002 192.168.1.253:7001 192.168.1.253:7002 --cluster-replicas 1

After successful startup, the results are as follows :

image-20211216185204745

From the known :

192.168.1.251:7001  master    its slave by  192.168.1.252:7002
192.168.1.252:7001  master    its slave by  192.168.1.253:7002
192.168.1.253:7001  master    its slave by  192.168.1.251:7002

Cluster operation

Landing in the cluster

 # -c, Log in as a cluster 
redis-cli -c -h 192.168.1.251 -p 7001 -a 123456        

View cluster information

#  State of the cluster 
192.168.1.251:7001> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:526
cluster_stats_messages_pong_sent:541
cluster_stats_messages_sent:1067
cluster_stats_messages_ping_received:536
cluster_stats_messages_pong_received:526
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1067

List node information

#  List node information 
192.168.1.251:7001> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639652443264 3 connected 5461-10922
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639652444267 5 connected 10923-16383
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639652442000 4 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639652441257 6 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] myself,master - 0 1639652442000 1 connected 0-5460
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639652442259 5 connected

Test data reading and writing

192.168.1.252:7001> set key1 one
OK

192.168.1.252:7001> set key2 211
-> Redirected to slot [4998] located at 192.168.1.251:7001
OK

192.168.1.253:7001> get key1
-> Redirected to slot [9189] located at 192.168.1.252:7001
"one"

192.168.1.252:7001> get key2
-> Redirected to slot [4998] located at 192.168.1.251:7001
"211"

It can be seen that redis cluster Clusters are decentralized , Every node is equal , You can get and set data by connecting any node .

slave Nodes don't provide services at all , Just as a counterpart master A backup of the node .

Add node

stay 192.168.1.251 To add 1 Nodes 7003

# vim /usr/local/redis-5.0.14/cluster/redis_7003.conf

bind 192.168.1.251
port 7003
daemonize yes
pidfile "/var/run/redis_7003.pid"
logfile "/usr/local/redis-5.0.14/cluster/redis_7003.log"
dir "/data1/redis/cluster/redis_7003"
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7003.conf
cluster-node-timeout 15000

# mkdir /data1/redis/cluster/redis_7003

# redis-server /usr/local/redis-5.0.14/cluster/redis_7003.conf 

stay 192.168.1.252 To add 1 Nodes 7003

# vim /usr/local/redis-5.0.14/cluster/redis_7003.conf

bind 192.168.1.252
port 7003
daemonize yes
pidfile "/var/run/redis_7003.pid"
logfile "/usr/local/redis-5.0.14/cluster/redis_7003.log"
dir "/data1/redis/cluster/redis_7003"
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7003.conf
cluster-node-timeout 15000

# mkdir /data1/redis/cluster/redis_7003

# redis-server /usr/local/redis-5.0.14/cluster/redis_7003.conf 

Add nodes to the cluster :

192.168.1.251:7001> CLUSTER MEET 192.168.1.251 7003
OK
192.168.1.251:7001> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639653625000 3 connected 5461-10922
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639653628000 5 connected 10923-16383
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639653627000 4 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639653629480 6 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] myself,master - 0 1639653626000 1 connected 0-5460
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] master - 0 1639653629000 0 connected
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639653628000 5 connected
192.168.1.251:7001> CLUSTER MEET 192.168.1.252 7003
OK
192.168.1.251:7001> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639653675000 3 connected 5461-10922
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] master - 0 1639653677000 7 connected
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639653677000 5 connected 10923-16383
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639653677000 4 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639653678644 6 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] myself,master - 0 1639653675000 1 connected 0-5460
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] master - 0 1639653677640 0 connected
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639653678000 5 connected

The newly added nodes are all based on master Join the cluster as

Change the node identity

Will add 192.168.1.252:7003 The node identity is changed to 192.168.1.251:7003 Of slave

redis-cli -c -h 192.168.1.252 -p 7003 -a 123456 cluster replicate 20f69b6053f05aec6b3a4e006115b19752c41a78
#  View the cluster status 
192.168.1.251:7001> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639654051000 3 connected 5461-10922
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] slave 20f69b6053f05aec6b3a4e006115b19752c41a78 0 1639654054906 7 connected
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639654053000 5 connected 10923-16383
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639654051000 4 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639654052898 6 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] myself,master - 0 1639654052000 1 connected 0-5460
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] master - 0 1639654053901 0 connected
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639654052000 5 connected
192.168.1.251:7001> 

Check out the corresponding nodes.conf file , Changes can be found , It records the node information of the current cluster

# cat /data1/redis/cluster/redis_7001/nodes_7001.conf
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] master - 0 1639653914000 1 connected 0-5460
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639653913000 4 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639653913076 6 connected
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] slave 20f69b6053f05aec6b3a4e006115b19752c41a78 0 1639653915080 7 connected
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] master - 0 1639653912000 0 connected
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639653912073 5 connected 10923-16383
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] myself,master - 0 1639653912000 3 connected 5461-10922
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639653914078 5 connected
vars currentEpoch 7 lastVoteEpoch 0

Redistribution hash Slot

#  Redistribution 
# redis-cli -c -h 192.168.1.251 -p 7001 -a 123456 --cluster rebalance --cluster-threshold 1 --cluster-use-empty-masters 192.168.1.251:7001
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 192.168.1.251:7001)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Rebalancing across 4 nodes. Total weight = 4.00
Moving 1366 slots from 192.168.1.252:7001 to 192.168.1.251:7003
######################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################
Moving 1365 slots from 192.168.1.253:7001 to 192.168.1.251:7003
#####################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################
Moving 1365 slots from 192.168.1.251:7001 to 192.168.1.251:7003
#####################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################
[[email protected] cluster]# 

simulation master Node down

192.168.1.251

[[email protected] redis_7001]# netstat -lntp |grep 7003
tcp        0      0 192.168.1.251:7003      0.0.0.0:*               LISTEN      13662/redis-server  
tcp        0      0 192.168.1.251:17003     0.0.0.0:*               LISTEN      13662/redis-server  

# kill 13662

You can see slave node 192.168.1.252:7003 Turned into master

192.168.1.252:7003> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639656794000 3 connected 6827-10922
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639656796000 5 connected
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] myself,master - 0 1639656794000 9 connected 0-1364 5461-6826 10923-12287
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639656796000 1 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639656795472 3 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] master - 0 1639656797480 1 connected 1365-5460
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639656796476 5 connected 12288-16383
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] master,fail - 1639656777897 1639656775391 8 disconnected

Now restart the test that just hung up master node

You can see master After the node is restored, it becomes slave node

192.168.1.252:7003> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639657537000 3 connected 6827-10922
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639657535264 5 connected
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] myself,master - 0 1639657537000 9 connected 0-1364 5461-6826 10923-12287
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639657537000 1 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639657539273 3 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] master - 0 1639657538270 1 connected 1365-5460
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639657537000 5 connected 12288-16383
20f69b6053f05aec6b3a4e006115b19752c41a78 192.168.1.251:[email protected] slave 16cd75f2b35423220330eee59ec6958e0d802b8f 0 1639657537268 9 connected

Delete node

Now I will add the two 7003 The node to delete

Delete first slave node

# redis-cli -c -h 192.168.1.251 -p 7001 -a 123456 --cluster del-node 192.168.1.251:7003 20f69b6053f05aec6b3a4e006115b19752c41a78

192.168.1.252:7003> CLUSTER NODES
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] master - 0 1639658067000 3 connected 6827-10922
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639658065000 5 connected
16cd75f2b35423220330eee59ec6958e0d802b8f 192.168.1.252:[email protected] myself,master - 0 1639658064000 9 connected 0-1364 5461-6826 10923-12287
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639658066000 1 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639658062000 3 connected
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] master - 0 1639658066000 1 connected 1365-5460
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639658067827 5 connected 12288-16383

Empty master Nodes in the hash Slot

take hash Slots are allocated to other master node

redis-cli -c -h 192.168.1.251 -p 7001  -a 123456 --cluster reshard 192.168.1.252:7003 --cluster-from 16cd75f2b35423220330eee59ec6958e0d802b8f --cluster-to e8ac4af0dd93b558260f1068b960d10df639ff5b --cluster-slots 1024 --cluster-yes

redis-cli -c -h 192.168.1.251 -p 7001  -a 123456 --cluster reshard 192.168.1.252:7003 --cluster-from 16cd75f2b35423220330eee59ec6958e0d802b8f --cluster-to e6db21fdc82d444e5958d446a66a9c76d0ea60cc --cluster-slots 1024 --cluster-yes

redis-cli -c -h 192.168.1.251 -p 7001  -a 123456 --cluster reshard 192.168.1.252:7003 --cluster-from 16cd75f2b35423220330eee59ec6958e0d802b8f --cluster-to 5d752d97d761a154989132841eabb1e12a46531f 

Delete master node

#redis-cli -c -h 192.168.1.251 -p 7001 -a 123456 --cluster del-node 192.168.1.252:7003 16cd75f2b35423220330eee59ec6958e0d802b8f 
 
 192.168.1.252:7001> CLUSTER NODES
e6db21fdc82d444e5958d446a66a9c76d0ea60cc 192.168.1.251:[email protected] master - 0 1639658525000 11 connected 1024-6143
63821c34dc1da011cd018731688e85a508391bdc 192.168.1.252:[email protected] slave e6db21fdc82d444e5958d446a66a9c76d0ea60cc 0 1639658525345 11 connected
2c763d241297ea8778f76d497632a9e8b7bf8e9b 192.168.1.253:[email protected] slave e8ac4af0dd93b558260f1068b960d10df639ff5b 0 1639658524344 10 connected
5d752d97d761a154989132841eabb1e12a46531f 192.168.1.253:[email protected] master - 0 1639658526348 12 connected 6144-6826 10923-16383
e8ac4af0dd93b558260f1068b960d10df639ff5b 192.168.1.252:[email protected] myself,master - 0 1639658525000 10 connected 0-1023 6827-10922
cd785a8263f97b26e1437095726a514e7f69d616 192.168.1.251:[email protected] slave 5d752d97d761a154989132841eabb1e12a46531f 0 1639658525000 12 connected

Node has been deleted , And cutting has stopped

copyright notice
author[Sun batian],Please bring the original link to reprint, thank you.
https://en.cdmana.com/2022/01/202201270145375135.html

Random recommended