Chanler

Chanler

HomeArchives

"Dark Horse Redis" II. Best Practices

14
AI Translation
This post is translated from Chinese into English through AI.View Original
AI-generated summary
The document discusses best practices for designing Redis keys, emphasizing a structured key format that enhances readability, avoids conflicts, and optimizes memory usage. It introduces the concept of "BigKey," highlighting potential issues such as network blocking and CPU pressure due to large key sizes or excessive member counts. Recommendations include keeping individual key values under 10KB and limiting collection types to fewer than 1000 elements. It also covers data structure choices for storing user objects, suggesting the use of strings or hashes with manageable entry limits. For large data imports, it advises using pipelines to reduce network overhead and improve efficiency. Regarding persistence, the document recommends disabling persistence for caching Redis instances, using AOF instead of RDB, and implementing regular backups on slave nodes. Deployment tips include ensuring sufficient memory for Redis instances and avoiding co-location with CPU-intensive or high disk-load applications. Lastly, it addresses slow queries, defining them as commands that exceed a specified execution time threshold, and provides configuration options for managing slow query logs.

Multi-level Cache Skipping

Key Design#

Elegant Key Structure#

Although Redis keys can be customized, it is best to follow the best practice conventions below:

  • Follow the basic format: [Business Name]:[Data Name]:[ID]
  • Length should not exceed 44 bytes
  • Should not contain special characters

For example: For our login business, saving user information, the key looks like this: login:user:10

Advantages:

  1. Strong readability
  2. Avoid key conflicts
  3. Easy management
  4. More memory-efficient: keys are of string type, and the underlying encoding includes int, embstr, and raw. Embstr is used for less than 44 bytes, utilizing contiguous memory space, resulting in lower memory usage.

set num 123 If type num returns string but object encoding num returns int at the underlying level.
set name Jack If type name returns string but object encoding name returns embstr at the underlying level.

BigKey#

What is BigKey?

  • The key itself has a large amount of data
  • Too many members
  • Large member data size

Recommended values:

  • The value of a single key should be less than 10KB
  • For collection-type keys, it is recommended that the number of elements be less than 1000

Hazards:

  1. Network blocking
  2. Data skew, uneven sharding
  3. Redis blocking
  4. High CPU pressure from serialization and deserialization

Appropriate Data Structure#

For storing a User object:

  1. Store the entire string as String type user:1
  2. Break down fields, store separately as String type user:1 user:1
  3. Use Hash structure

Hash structure should not exceed 1000 entries and can be split.

Batch Processing#

Large Data Import#

1 command response = 1 network round trip + 1 execution

Therefore, it is better to transmit once rather than multiple times for the same amount of data; be careful not to overload, or it will block the network.

Pipeline commands into the pipeline, supporting various native commands, simply sends all these commands to Redis at once, rather than the atomicity of native MSET for large operations.

Persistence Configuration#

Although Redis persistence can ensure data safety, it also brings many additional overheads. Therefore, please follow the suggestions below for persistence:

  1. Redis instances used for caching should avoid enabling persistence features.
  2. It is recommended to disable RDB persistence and use AOF persistence.
  3. Use scripts to periodically perform RDB on slave nodes for data backup.
  4. Set reasonable rewrite thresholds to avoid frequent bgrewrite.
  5. Configure no-appendfsync-on-rewrite = yes to prohibit AOF during rewrite, avoiding blocking caused by AOF.

Deployment-related suggestions:

  1. The physical machine for the Redis instance should reserve enough memory to handle fork and rewrite.
  2. The memory limit for a single Redis instance should not be too large, such as 4G or 8G. This can speed up the fork process and reduce master-slave synchronization and data migration pressure.
  3. Do not deploy alongside CPU-intensive applications.
  4. Do not deploy alongside applications with high disk load, such as databases or message queues.

Slow Queries#

Executing commands in Redis that take longer than a certain threshold is called a slow query (not just queries). Redis is single-threaded, so slow queries will block.

The threshold for slow queries can be specified through configuration:

  • slowlog-log-slower-than: Slow query threshold, in microseconds. The default is 10000, recommended to be 1000.

Slow queries will be recorded in the slow query log, which has a length limit that can be specified through configuration:

  • slowlog-max-len: Length of the slow query log (essentially a queue). The default is 128, recommended to be 1000.

To view the slow query log list:

  • slowlog len: Query the length of the slow query log.
  • slowlog get [n]: Read [n]/all slow query logs.
  • slowlog reset: Clear the slow query list.

This article is synchronized and updated to xLog by Mix Space. The original link is https://blog.0xling.cyou/posts/redis/redis-2

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.