In the process of using redis, expired keys will inevitably be generated, and these keys will not be deleted immediately in real time after they expire. When the number of these keys accumulates more and more, it will occupy a lot of memory. Therefore, three strategies are used at the bottom of redis to delete these keys.
The first strategy: passive deletion
When reading/writing a key, redis will first check whether the key exists. If it exists and has expired, delete the key directly and returnnilTo the client.
The second strategy: periodic deletion
There is a series of regular tasks (serverCron) in redis. These tasks run every once in a while, including tasks to clean up expired keys, and the running frequency is from the configuration file.hzControlled by parameters, the value range is 1~500, and the default is 10, which means running 10 times per second.
The cleaning process is as follows:
- Iterate through all dbs
- Random check 20 keys from a collection of keys with expiration time set in db
- Delete all expired keys found in the check
- If more than 25% of the keys in the check result have expired, continue to repeat steps 2-3, otherwise continue to traverse the next db
IncreasehzIt will increase the frequency of redis regular tasks. If your redis contains many expired keys, you can consider increasing this value, but be careful to increase the CPU pressure. The author of redis recommends that this value should not exceed 100.
The third strategy: Forced deletion
If the memory used by redis has reachedmaxmemoryWhen the configured value is triggered, a forced cleanup policy will be triggered, and the cleanup policy will be generated by the configuration file.maxmemory-policyParameters to control
There are these cleaning strategies:
-
volatile-lru: Use the LRU algorithm to clean up the keys with expiration time (default value) -
allkeys-lru: Use the LRU algorithm to clean all keys -
volatile-lfu: Use the LFU algorithm to clean up the keys with expiration time (redis 4.0 version has started to support it) -
allkeys-lfu: Use the LFU algorithm to clean all keys (redis 4.0 version has started to support it) -
volatile-random: Random cleaning of all keys with expiration time set -
allkeys-random: Random cleaning from all keys -
volatile-ttl: Clean up the key with the smallest survival time -
noeviction: No cleaning is done, and all write operations are refused (if you need to ensure the integrity of the data, you can choose this)
In order to save memory and performance considerations, the above cleaning strategy does not need to traverse all data, but uses a random sampling method to randomly fetch a specific number at a time (bymaxmemory-samplesConfiguration item controls, default is 5 keys), then execute the LRU algorithm, RANDOM algorithm, or find the key with the smallest TTL time and then delete it.
Note: This cleaning process is blocked and will not stop until enough memory space is cleaned.
About the cleaning of big key
When deleting a collection with a large number of elements (set/hash/list/sortedSet), whether it is usedDELCommand deletion or redis deletion to free up memory space. When deleting these big keys, the main thread of redis will block. To solve this problem, in redis 4.0, the feature of lazy free is provided.
When using lazy free to delete a big key, it will return in sub-milliseconds just like the time-consuming process of an O(1) instruction, and then hand over the time-consuming operation of truly deleting the key to the bio background child thread to execute.
UNLINK Command
UNLINKThe command is withDELThe lazy free implementation of the key function is also removed.
The only difference is that when UNLINK deletes a key of a collection type, if the number of elements of the set key is greater than 64, the real memory release operation will be handed over to a separate background thread to operate. Example:
127.0.0.1:6379> UNLINK mylist (integer) 1
FLUSHALL/FLUSHDB command
FLUSHALL/FLUSHDBThe command also has a lazy free implementation, and add it after the command.ASYNCJust keywords, use examples:
127.0.0.1:6379> FLUSHALL ASYNC
lazy free related configuration items
The configuration items related to lazy free have the following, and the default values are no, that is, turn off.
lazyfree-lazy-eviction
Redis memory usagemaxmemory, and when setting up an elimination strategy, whether to use the lazy free mechanism when eliminating the key.
Note: If lazy free is enabled in this scenario, the memory release of the eliminated key may be untimely, resulting in redis not being able to quickly reduce memory usage below maxmemory.
lazyfree-lazy-expire
For keys with expiration time set, whether to use the lazy free mechanism when they are cleaned and deleted by Redis after expiration is reached, it is recommended to enable this scenario.
lazyfree-lazy-server-del
For some commands, when processing existing keys, they will have an implicit DEL key operation. likeRENAMECommand, when the target key already exists, redis will first delete the target key. If these target keys are a big key, blocking performance problems will occur. This parameter setting is to solve this problem, and it is recommended to enable it.
slave-lazy-flush
All data synchronization is performed for slave. The slave will run before loading the master's RDB file.FLUSHALLCome and clean up your own data scene.
Parameter settings determine whether to use the lazy free flush mechanism. If the memory does not change much, it is recommended to enable it. It can reduce the time spent in full synchronization, thereby reducing the memory usage growth caused by the main library due to the explosion of output buffers.
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.