Interviewer: how do you choose the redis persistence mechanism AOF log and RDB snapshot at work

Everybody knows Redis Often used in cached scenes , Have you ever thought about such a question , Once the server goes down , All data in memory is lost , How do we recover ？ If you restore directly from the back-end database , It will not only bring great pressure to the database , It also slows down the response of the upper application . therefore redis The persistence mechanism is very important . Let's talk about Redis Persistence mechanism of . at present Redis There are two main mechanisms for persistence , namely AOF(Append Only File) Journal and RDB snapshot . Next, let's learn about .

AOF journal

AOF journal , Post write log , It means Redis Execute the order first , Write data to memory , Then write to the log .Redis Why execute the command first and then write to the log ？ Let's look at it first AOF What is recorded in the log .AOF The record is Redis Every order you receive , These logs are saved as text . If we do set hello world command ,AOF Is as follows :

among “*3” Represents that the current command has 3 part , Each part is made up of ”$+ Numbers ” start , Followed by specific orders 、 Key and value . The number here represents the following command 、 How many bytes are the key and value in total . for example “$3 set” This part has 3 Bytes , That is to say “set” command .

Redis To avoid the extra overhead of checking , Go back to AOF When writing logs in , There is no syntax check for these commands . So if you write the log first and then execute the command , Some wrong commands may be recorded in the log ,Redis When using log recovery, there will be an error . And the way to write a post log , Is to execute the order first , If there is an error in command execution , Do not write to the log , Only successful commands are written to the log . therefore Redis One of the great advantages of using post write logging is that , It can avoid recording wrong commands .AOF There's another big advantage , It logs after the command is executed , So it doesn't block the current write operation .

however ,AOF There are also two potential risks , First, if you have just executed a command , It went down before I could write a log , Then the command and the corresponding data are at risk of loss . secondly ,AOF Although it avoids blocking the current command , But there may be blocking risks for the next operation . This is because ,AOF The log is also executed in the main thread , How to write log files to disk slowly , Will block subsequent operations . In response to this question ,AOF It provides us with three writeback strategies , That is to say AOF Configuration item for appendfsync Three optional values for .

1. Always, Synchronous write back ： Each write command is executed , Write the log back to disk immediately and synchronously ;
2. Everysec, Write back every second ： Each write command is executed , Just write the log first AOF Memory buffer for files , Write the contents of the buffer to disk every second ;
3. No, Write back of operating system control ： Each write command is executed , Just write the log first AOF Memory buffer for files , It's up to the operating system to decide when to write the contents of the buffer back to disk .

None of the three strategies can achieve the best of both worlds , All have their own advantages and disadvantages , We can only according to our business scenario , Whether you need high performance or high reliability to choose different writeback strategies .
 

Last ,AOF There's another problem , Namely AOF Record on disk in the form of a file . With Redis More and more write commands are accepted , that AOF The log file will become larger and larger , So we need to take some measures to control AOF The size of the log file . This is the time ,AOF The rewriting mechanism comes in handy .AOF Rewriting mechanism means that when rewriting ,Redis A new database will be created according to the current situation of the database AOF file , in other words , Read all key value pairs in the database , Then for each key value pair, a command is used to record its writing . Why can rewriting mechanism make log files smaller ? actually , The rewriting mechanism has “ Changeable one ” function . So-called “ Changeable one ”, in other words , Multiple commands in old log file , In the rewritten new log, it becomes a command . For example, we are interested in a certain key the 6 Write operations , Then there will be... In the old log file 6 Bar record , There is only one command in the rewritten log file , therefore AOF Rewriting will reduce the size of the file . that AOF Does rewriting block the main thread ？ After all, put the whole Redis The operation logs of the latest data of the database are written back to disk , It's still a very time-consuming process . and AOF The log is written back by the main thread , The rewriting process is made up of a background subprocess bgrewriteaof To complete , This is also to avoid blocking the main thread , Lead to Redis Performance degradation of . Every time an override is executed , The main thread fork Backstage bgrewriteaof Subprocesses . here ,fork Will copy the memory of the main thread （ Using the write time replication technology of the operating system , No real copy , Copy on write technology will be analyzed below ） One for bgrewriteaof Subprocesses , This contains the latest data in the database . then ,bgrewriteaof The child process can be used without affecting the main thread , Write the copied data into operations one by one , Write in the rewrite log . Because the main thread is not blocked , Can still handle new operations . To avoid data loss , stay AOF In the process of rewriting , The newly entered write command will be written to two logs . The first log is in use AOF journal ,Redis Will write the operation to its buffer . thus , Even if it goes down , This AOF Log operations are still complete , Can be used to restore . The second log , It means new AOF Rewrite log . This operation is also written to the buffer of the rewrite log . such , Rewriting the log will not lose the latest operation . Wait until all operation records of the copied data are rewritten , These latest operations that rewrite the log record also write new ones AOF file , To ensure the latest state of the database records . here , We can use the new AOF The document replaced the old one .

RDB Snapshot file

because AOF What is recorded is the operation command , Not the actual data . therefore , use AOF Method for fault recovery , You need to execute the operation logs one by one . If there are many operation logs , that Redis Recovery is slow , Affect normal use . Does that guarantee reliability , It can also achieve rapid recovery in case of downtime ？ That's the memory snapshot . about Redis Come on , It writes the state of a certain time to the disk in the form of a file . thus , Even if it goes down , Snapshot files will not be lost , The reliability of the data is guaranteed . This snapshot file is called RDB（Redis DataBase） file . and AOF comparison ,RDB It records the data at a certain time , It's not the operation , therefore , When doing data recovery , Put... Directly RDB The file is loaded into memory , Quickly complete the recovery .

Redis Two commands are provided to generate RDB file , Namely save and bgsave.

1. save： Execute... In the main thread , It can cause congestion .
2. bgsave： Create a subprocess , Specifically for writing RDB file , Avoid blocking the main thread . This is also redis Generate RDB The default configuration of the file .

therefore , We can use bgsave To perform a full snapshot , It not only ensures the reliability , Again, avoid Redis Performance impact . Next , Let's think about such a problem , Namely Redis Take a full snapshot ,Redis Can the data in be modified ？Redis Do you still support write operations ？ Pause write for snapshot ,Redis It must be unacceptable . therefore Redis With the help of the write time replication technology provided by the operating system （Copy-On-Write,COW）. Simply speaking ,bgsave The child process is made up of the main thread fork Generated , All memory data of the main thread can be shared .bgsave After running , Start reading memory data in the main thread , And write them in RDB file . here , If the main thread reads these data , Then the main thread and bgsave Subprocesses don't interact with each other . however , If the main thread performs a write or modify operation , That is to modify a piece of data in memory , Then this data will be copied , Make a copy . Then the main thread makes changes on the copy . meanwhile ,bgsave The subprocess continues to write the original data to RDB file . This ensures the integrity of the snapshot , It also avoids the impact on normal business .

Next, let's look at the next question , Is how often to take a snapshot , If the snapshot interval is too long , The more data you lose , The interval is too short , The less data is lost , However, frequent full snapshots can put a lot of pressure on the disk . because fork Create child process bgsave This process needs to block the main thread , The more memory the main thread has ,fork The longer the time . So often fork Out bgsave Subprocesses , It will frequently block the main thread . What's a good way to make use of RDB Fast recovery of , At the same time, it can reduce the loss of data with a small cost .Redis4.0 A hybrid AOF Journal and RDB Methods . Simply speaking , Memory snapshots are executed at a certain frequency , Use... Between snapshots AOF Record the operation command .

So to conclude , About AOF and RDB The choice of , To provide you with 3 Some suggestions .

1. When data cannot be lost , Memory snapshot and AOF It's a good choice to use a mixture of .
2. If minute level data loss is allowed , You can just use RDB.
3. If only AOF, priority of use everysec Configuration options , Because it strikes a balance between reliability and performance .