current position:Home>Redis basis

Redis basis

2022-08-06 17:40:01tender.dj

Quick reason:存储内存、io多路复用、单线程;

1.基本数据结构

五种:string、list、hash、set、zset(5.0加入stream)

string

底层数据结构:动态字符串(sds),redis基于c开发,字符数组为char[],redis没有采用这种,自己构建了一个结构体struct{int len;int free;char[] buf},有点类似go的slice底层,go存的是一个指针;

sdsGetting the length is faster(直接len字段o(1),c为o(n)),Buffer overflow can be prevented,有free字段,There is enough space for direct splicing,不够再扩容;Expansion is less than1M翻倍,大于1M,每次加1M;The memory reallocation caused by modification can be reduced(Freeing up space is not really freeing,而是设置free值);

应用:计数、分布式锁(写lua脚本,保证setnx,setex原子操作);

list

底层:小于512压缩列表,Otherwise doubly linked list;

压缩列表:连续的内存,包括多个entry;

struct entry {

int prelen; 前一个元素的字节长度

int encoding;

byte[] content; content of the element

}

struct ziplist {

int bytes; The number of bytes occupied by the entire compressed list

int tail_offset; The number of bytes the last element is from the start of the packed list, Easy to traverse in reverse

Int length; 元素总数

T[] entires; 元素列表

int end; 压缩列表结束标志

}

双端链表:The pointer will compress the linked list connection;

struct quickListNode {

        quickListNode* prev; Points to the previous compressed list

        quickListNode* next; Points to the latter compressed list

        ziplist* p; Points to the current compressed list header

        int count; The total number of elements in the current packed list

        int size; The total number of bytes occupied by the current compressed list

}

struct quickList {

        quickListNode* head; Quick list header

        quickListNode* tail; The end of the quick linked list

        int count; 元素总数

        int nodes; The number of compressed lists
}

参考:https://blog.csdn.net/jinixin/article/details/110004723?spm=1001.2014.3001.5502

应用:消息队列

hash

底层:小于512压缩列表,Otherwise a dictionary(哈希表实现,A dictionary is implemented by two hash tables,One for progressiverehash);

typedef struct dict {
    dictType *type;
    void *privdata;
    // 哈希表
    dictht ht[2];
    //rehash 索引,当 rehash 不在进行时,值为 -1
    int rehashidx; /* rehashing not in progress if rehashidx == -1 */
    // 目前正在运行的安全迭代器的数量
    int iterators; /* number of iterators currently running */
} dict;

应用:Mainly used to store objects,注意大key问题;

set

底层:小于512,intset(整数集合,All consist of integers),Otherwise a dictionary(val为空);

typedef struct intset {   
    // 编码方式
    uint32_t encoding;
    // 集合包含的元素数量
    uint32_t length;
    // 保存元素的数组 按照从小到大的顺序,且不重复
    int8_t contents[];
} intset;

应用:公共好友(setCan be handed over and operated)

zset

底层:Data compaction list,Use dictionaries+跳表,查找o(1),排序,范围获取o(logn);

Dictionary storage format:key score member(value),skip tablescore并以此排序

struct zset {

        dict A; 所有val=>score对

        skiplist A; 跳跃列表

}

struct skiplistnode {

        string value;

        double score;

        skiplistnode*[] forwards; Multiple layers of pointers

}

struct skiplist {

        skiplistnode* header; 首元素指针

        int maxLevel; The current highest level

}

Range lookup can directly find the first element,Afterwards, the subsequent elements can be returned directly according to the level;

应用:排行榜(游戏top排名,字段+时间戳排序,You can combine fields to be sorted and timestamps intoscore);

分布式一致性问题:

延迟双删:先删缓存,再更新数据库,Delete the cache after a while(Business reading time decision);

redis持久化:

rdb:快照(n分钟m次记录),Binary writes to disk,Rebooting will lose data,但相比于aof快,fork子进程,共享内存,Execute business while synchronizing,A record will be lost;

aof:追加到file尾部,Reboot to replay,耗时,不会丢数据;

rdb+aof:两者结合,aofFull logs are not stored,Saved after the last persistence,重启时,Load the data from the snapshot first,然后重放增量;

redis主从

Divided into full and incremental synchronization,为非阻塞,The first time is full synchronization;

Full steps:

1.slave连接master,发送sync(fsync)命令;

2.master收到sync命令后,执行bgsave命令保存rdb快照文件,At the same time, the buffer area is used to save all write commands executed afterward;

3.master执行bgsave后,向所有slave发送rdb文件,Write commands are also logged during transmission;

4.slave收到快照后,丢弃旧数据,Load a new snapshot;

5.masterAfter sending the snapshot, continue to send the write command to the buffer area;

6.slaveExecute the write command to the buffer area;

增量步骤:

1.slave发slave_repl_offset,Sync with offset position,为增量;

2.master执行bgsave,发送offsetafter the write command;

redis异常

击穿:redis没有,数据库有,单keyHotspot keeps requesting,Hit the database directly when it expires,造成数据库压力过大,请求阻塞;

解决:1.热点数据永不过期;2.加互斥锁(Distributed systems use distributed locks),key过期时,请求打到数据库,Open a mutex,It is guaranteed that only one process can access the current one at a timekey;

穿透:redis没有,数据库也没有,Attackers keep requesting data that doesn't exist,This results in a large number of requests hitting the database in a short period of time,dbwill be too stressful;

解决:1.布隆过滤器(Based on bitmaps and hash functions,marray of bits+k个哈希函数,To reach the judgment that it does not exist must not exist,The presence of data does not necessarily have effects);There is a problem that the Bloom filter cannot be deleted,The cuckoo filter can play a certain optimization effect;2.redis存储空对象,设置过期时间;

雪崩:大批量的key过期,Reaching the database causes downtime;

解决:设置随机过期时间,均匀过期;

redis key过期

1.惰性过期:一个keyThe expiration time is judged when it is used,如果过期则删除;内存不友好,if one expireskeyNot used for a long time,会占用内存;cpu友好,只有cpuDeleted only by operation;

2.主动过期:Regular random scan part of the set expiration timekey,进行删除;

3.定期过期(两者结合,redis采用):The expiration time will be setkey放入字典,每100ms扫描一次,Random scan draw20个,Maximum time per scan25ms,防止卡顿;

从库过期策略:Handle expiration from the librarykeyfor passive,Does not expire scans;主库的key过期,会生成del指令到aof文件,然后同步给从库,Execute from library passdel进行删除;Instructions are synchronous to asynchronous,如果不及时,会造成主从不一致,The main library does not have itkeyAlso exists in the slave library;

redis内存淘汰

1.noeviction:Only read and delete operations are supported,Write service error;

2.allkeys-xxx:ttl:Expiration time is small to delete;random:随机删除;lru:Least recently used delete;lfu:Delete is used least often;

3.volatile-xxx:ttl:Expiration time is small to delete;random:随机删除;lru:Least recently used delete;lfu:Delete is used least often;

redis集群模式

主从

哨兵

分片

copyright notice
author[tender.dj],Please bring the original link to reprint, thank you.
https://en.cdmana.com/2022/218/202208061719267204.html

Random recommended