redis内存分析工具redis-rdb-tools

共计 7744 个字符，预计需要花费 20 分钟才能阅读完成。

分析redis内存数据的使用情况，如bigkeys，hotkeys等

redis-rdb-tools分析工具

工具安装

# 安装python3环境
[root@redis01 ~]# yum install python3 gcc gcc-c++ libffi-devel python3-devel -y

# 安装rdbtools工具
[root@redis01 ~]# pip3 install rdbtools python-lzf

# 安装完后有如下三个工具
[root@redis01 ~]# which rdb redis-profiler redis-memory-for-key
/usr/local/bin/rdb
/usr/local/bin/redis-profiler
/usr/local/bin/redis-memory-for-key

redis测试数据生成

# 使用如下脚本自动生成简单str类型数据
[root@redis01 ~]# cat test.sh
#!/bin/bash
for i in {1..10000}
do
  myvalue=`head /dev/urandom | tr -dc A-Za-z0-9 | head -c $RANDOM`
  #echo "key${i} ${value}"
  echo "key${i}"
  redis-cli -c -h 127.0.0.1 -p 6379 set key${i} ${myvalue}
done

工具用法

• rdb：用于解析rdb文件
• redis-memeory-for-key：解析server里单个key
• redis-profiler：解析rdb文件成HTML页面（该页面中js/css引用的是国外资源，所以打开会出现样式资源加载超时而渲染失败）

[root@redis01 ~]# rdb --help
usage: usage: rdb [options] /path/to/dump.rdb

Example : rdb --command json -k "user.*" /var/redis/6379/dump.rdb

positional arguments:
  dump_file             RDB Dump file to process

optional arguments:
  -h, --help            show this help message and exit
  -c CMD, --command CMD
                        Command to execute. Valid commands are json, diff,
                        justkeys, justkeyvals, memory and protocol
  -f FILE, --file FILE  Output file
  -n DBS, --db DBS      Database Number. Multiple databases can be provided.
                        If not specified, all databases will be included.
  -k KEYS, --key KEYS   Keys to export. This can be a regular expression
  -o NOT_KEYS, --not-key NOT_KEYS
                        Keys Not to export. This can be a regular expression
  -t TYPES, --type TYPES
                        Data types to include. Possible values are string,
                        hash, set, sortedset, list. Multiple typees can be
                        provided. If not specified, all data types will be
                        returned
  -b BYTES, --bytes BYTES
                        Limit memory output to keys greater to or equal to
                        this value (in bytes)
  -l LARGEST, --largest LARGEST
                        Limit memory output to only the top N keys (by size)
  -e {raw,print,utf8,base64}, --escape {raw,print,utf8,base64}
                        Escape strings to encoding: raw (default), print,
                        utf8, or base64.
  -x, --no-expire       With protocol command, remove expiry from all keys
  -a N, --amend-expire N
                        With protocol command, add N seconds to key expiry
                        time

# 查看单key分析
[root@redis01 ~]# redis-memory-for-key --help
Usage: redis-memory-for-key [options] redis-key
Examples :
redis-memory-for-key user:13423
redis-memory-for-key -s localhost -p 6379 user:13423


Options:
  -h, --help            show this help message and exit
  -s HOST, --server=HOST
                        Redis Server hostname. Defaults to 127.0.0.1
  -p PORT, --port=PORT  Redis Server port. Defaults to 6379
  -a PASSWORD, --password=PASSWORD
                        Password to use when connecting to the server
  -d DB, --db=DB        Database number, defaults to 0

# 生成内存分析html页面
[root@redis01 ~]# redis-profiler --help
Usage: redis-profiler [options] /path/to/dump.rdb

Example 1 : redis-profiler -k "user.*" -k "friends.*" -f memoryreport.html /var/redis/6379/dump.rdb
Example 2 : redis-profiler /var/redis/6379/dump.rdb

Options:
  -h, --help            show this help message and exit
  -f FILE, --file=FILE  Output file
  -k KEYS, --key=KEYS   Keys that should be grouped together. Multiple regexes
                        can be provided

常用操作

# 按json格式导出rdb
rdb --command json dump.rdb

# 导出rdb中的keys
rdb -c justkeys dump.rdb|uniq

# 导出rdb中的values
rdb -c justkeyvals dump.rdb

# 导出指定数据库的keys
rdb -c justkeyvals dump.rdb -n 0

# 导出匹配（正则）的keys
rdb --command justkeyvals --key ".*set*" dump.rdb

# 不导出匹配（正则）的keys
rdb --command justkeyvals --not-key ".*set*" dump.rdb

# 导出大于指定字节的keys
rdb --command memory --bytes 128  dump.rdb

# 导出内存字节排名前3个keys
rdb --command memory --largest 3 dump.rdb

# 导出持久keys数据
rdb --command memory --no-expire dump.rdb

# 导出指定时间内过期的数据
rdb --command memory --amend-expire 100 dump.rdb 

分析bigkeys

bigkeys的影响

bigkeys主要影响：

• 堵塞网络：因为单keys过大，并发量上去后带宽压力很明显
• 超时堵塞：因为key占用空间大，redis操作时效率低，容易造成性能影响
• 容易导致集群内存空间不平衡：如果某类key在同一个节点，则很容易造成该节点内存爆满

如何判断是不是bigkeys?

• 一个ZSET类型的Key，它的成员数量为10000个（成员数量过多）
• 一个HASH格式的Key，它的成员数量虽然只有1000个但这些成员的value总大小为100MB（成员体积过大）
• 一个STRING类型的Key，它的值为5MB（数据过大）一个LIST类型的Key，它的列表数量为20000个（列表数量过多）

上面的数据仅供参考，大Key的判定需要根据Redis的实际使用场景、业务场景来进行综合判断得出自己业务的合理阈值

redis-cli查看bigkeys

[root@redis01 redis]# redis-cli --bigkeys

# Scanning the entire keyspace to find biggest keys as well as
# average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).

[00.00%] Biggest string found so far 'key1231' with 432 bytes
[00.00%] Biggest string found so far 'key51843' with 559 bytes
[00.00%] Biggest string found so far 'key38379' with 795 bytes
[00.00%] Biggest string found so far 'key94258' with 853 bytes
[00.01%] Biggest string found so far 'key59606' with 894 bytes
[00.03%] Biggest string found so far 'key36221' with 1230 bytes
[00.08%] Biggest string found so far 'key66581' with 1335 bytes
[00.25%] Biggest string found so far 'key37315' with 1418 bytes
[00.27%] Biggest string found so far 'key95567' with 1585 bytes
[05.68%] Biggest string found so far 'key53088' with 1651 bytes
[45.08%] Biggest list   found so far 'mylist' with 20100 items
[69.66%] Biggest string found so far 'key743' with 1800 bytes
[82.32%] Biggest hash   found so far 'myset:__rand_int__' with 1 fields
[85.67%] Biggest string found so far 'key17820' with 1812 bytes

-------- summary -------

Sampled 100536 keys in the keyspace!
Total key length in bytes is 793727 (avg len 7.89)

Biggest string found 'key17820' has 1812 bytes
Biggest   list found 'mylist' has 20100 items
Biggest   hash found 'myset:__rand_int__' has 1 fields

100534 strings with 61694953 bytes (100.00% of keys, avg size 613.67)
1 lists with 20100 items (00.00% of keys, avg size 20100.00)
0 sets with 0 members (00.00% of keys, avg size 0.00)
1 hashs with 1 fields (00.00% of keys, avg size 1.00)
0 zsets with 0 members (00.00% of keys, avg size 0.00)

rdb生成内存表格报告查看bigkeys

[root@redis01 redis]# pwd
/var/lib/redis
[root@redis01 redis]# ls -l
总用量 63272
-rw-r--r--. 1 redis redis       53 5月  20 02:17 appendonly.aof
-rw-r--r--. 1 redis redis 62892201 5月  20 03:17 dump.rdb
[root@redis01 redis]# rdb -c memory dump.rdb --bytes 128 -f memory.csv
[root@redis01 redis]# ls -l
总用量 63272
-rw-r--r--. 1 redis redis       53 5月  20 02:17 appendonly.aof
-rw-r--r--. 1 redis redis 62892201 5月  20 03:17 dump.rdb
-rw-r--r--. 1 root  root   1318880 5月  20 02:59 memory.csv

• database：数据库编号
• type：数据类型
• key：建名
• size_in_bytes：使用的内存（键、值等）
• encoding：RDB编码类型
• num_elements：key中的value个数
• len_largest_element：key中的value的长度
• expiry：过期时间

对比RDB文件

博主之前遇到线上进行业务性能压测，实例A性能规格不够，需要一个更高规格的实例来承担此次压测，遂新建一个实例B，然后用开发的预热脚本刷数据，结果发现实例B的压测结果远远不如实例A，多次压测均是这种结果。难道数据未预热完全？多次压测，第一未预热全，第二次数据也会从数据库中缓存到redis，而且开发表面预热脚本没问题，实例A也是这么刷上去的。对比两个实例，实例A的QPS是实例B的一半，而且用实例B压测时发现多次都存在后端数据库查询。排除后端数据库查询语句发现查询的数据并没有找到，所以不会将数据缓存到B，而这些查询是上一个版本应用逻辑，此次版本不应该存在这些逻辑。而实例A做压测也未曾清空数据库后再压，所以每次版本的一些留存数据都会有，而实例B是新建的则没有。最后我们将实例A的数据导入到实例B中后，确实性能得到飞升。

从上面这件事里可以得出：1.压测需要持续保持干净的环境。2.实例数据迁移需要校验数据是否对等。3.应用版本逻辑检查

# 使用rdb和diff检测连个rdb文件差异
[root@redis01 redis]# rdb --command diff dump1.rdb | sort > dump1.txt

# 新增测试数据
[root@redis01 redis]# redis-cli
127.0.0.1:6379> bgsave
Background saving started
127.0.0.1:6379> set a 2
OK
127.0.0.1:6379> set b 3
OK
127.0.0.1:6379> exit

[root@redis01 redis]# rdb --command diff dump2.rdb | sort > dump2.txt
[root@redis01 redis]# diff dump1.txt dump2.txt
1c1,2
< db=0 a -> sdfs
---
> db=0 a -> 2
> db=0 b -> 3

生成内存HTML报告

[root@redis01 redis]# mredis-profiler dump.rdb -f redis.html
[root@redis01 redis]# myum install nginx -y
[root@redis01 redis]# msystemctl start nginx
[root@redis01 redis]# mmv redis.html /usr/share/nginx/html/
# 直接打开你的nginx服务地址访问该页面,页面css/js资源来自国外网站，所以需要梯子~

hotkeys分析

• 某Redis实例的每秒总访问量为10000，而其中一个Key的每秒访问量达到了7000（访问次数显著高于其它Key）
• 对一个拥有上千个成员且总大小为1MB的HASH Key每秒发送大量的HGETALL（带宽占用显著高于其它Key）
• 对一个拥有数万个成员的ZSET Key每秒发送大量的ZRANGE（CPU时间占用显著高于其它Key）

以上数据仅供参考，需要结合自己业务场景指定合理的阈值

使用redis-cli分析hotkeys，此命令在4.0版redis提供

[root@redis01 redis]#
 redis-cli --hotkeys
# Scanning the entire keyspace to find hot keys as well as
# average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).

[00.00%] Hot key 'counter:000000000002' found so far with counter 87
[00.00%] Hot key 'key:000000000001' found so far with counter 254
[00.00%] Hot key 'mylist' found so far with counter 107
[00.00%] Hot key 'key:000000000000' found so far with counter 254
[45.45%] Hot key 'counter:000000000001' found so far with counter 87
[45.45%] Hot key 'key:000000000002' found so far with counter 254
[45.45%] Hot key 'myset' found so far with counter 64
[45.45%] Hot key 'counter:000000000000' found so far with counter 93

-------- summary -------

Sampled 22 keys in the keyspace!
hot key found with counter: 254    keyname: key:000000000001
hot key found with counter: 254    keyname: key:000000000000
hot key found with counter: 254    keyname: key:000000000002
hot key found with counter: 107    keyname: mylist
hot key found with counter: 93    keyname: counter:000000000000
hot key found with counter: 87    keyname: counter:000000000002
hot key found with counter: 87    keyname: counter:000000000001
hot key found with counter: 64    keyname: myset