开发使用redis和Jedis注意事项

╰半橙微兮° 2022-01-22 05:09 451阅读 0赞

redis虽然是高性能，如果使用不当也极容易导致不可预知的生产故障。下面将从redis的使用上加以规避，主要还是细化开发者使用规范。

### 键值规范 ###

<table> 
 <tbody> 
  <tr> 
   <td style="width:84px;"> <p>key</p> </td> 
   <td> <p>【建议】可管理性：业务名(或数据库名)为前缀(防止key冲突)，用冒号分隔，比如业务名:表名:id</p> <p>【建议】简洁性：控制key的长度，太长内存占用也不容忽视</p> <p>【强制】特殊字符：不能保护包含空格、换行、单双引号以及其他转义字符</p> <p>【强制】类型：必须为string，我见过序列化时不注意，导致前面存在特殊null字符</p> </td> 
  </tr> 
  <tr> 
   <td style="width:84px;"> <p>value</p> </td> 
   <td> <p>【强制】简洁性：string类型控制在10KB以内，hash、list、set、zset元素个数不要超过5000，否则会导致防止网卡流量、慢查询</p> <p>【建议类型：选择适合的数据类型，比如ziplist可以配置控制内存编码优化，要注意节省内存和性能之间的平衡</p> </td> 
  </tr> 
 </tbody> 
</table>

### 生命周期 ###

【强制】使用expire设置过期时间，但最后打散过期时间，防止集中过期

### keys & flushall & flushdb ###

禁止线上使用keys、flushall、flushdb等，通过redis的rename机制禁掉命令。

### setnx ###

分布式锁并非完美，具体可参阅[Redis 分布式锁][Redis]。![90897223][]

### del ###

比如有一个数量比较大key（bigkey），不要使用del删除，scan的方式**渐进式删除（**hscan、sscan、zscan），同时要注意防止bigkey过期时间自动删除问题(会触发del操作，造成阻塞，而且该操作不会不出现在慢查询中(latency可查))

注：redis 4.0已经支持key的异步删除，建议升级使用

### select ###

redis的多数据库较弱，使用数字进行区分，很多客户端支持较差，同时多业务用多数据库实际还是单线程处理，会有干扰，因此使用一定要非常注意。

### mget & mset ###

原生命令使用批量操作（mget、mset）提高效率

注意：要控制一次批量操作的元素个数

### pipeline ###

非原生命令：可以使用批量操作（pipeline）提高效率

**原生命令和非原生命令**

*  mget的原生是原子操作
 *  pipeline可以打包不同的命令，原生做不到
 *  pipeline需要客户端和服务端双方同时支持

注意：要控制一次批量操作的元素个数

### 事务 ###

Redis事务功能较弱，不建议过多使用。Redis的事务功能较弱(不支持回滚)，而且集群版本(自研和官方)要求一次事务操作的key必须在一个slot上(可以使用hashtag功能解决)。

### Lua脚本 ###

lua脚本也是为了实现原子性操作，但原子操作Redis集群版本在使用Lua上有特殊要求：

*  所有key都应该由 KEYS 数组来传递，redis.call/pcall 里面调用的redis命令，key的位置，必须是KEYS array, 否则直接返回error，"-ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS array\\r\\n"
 *  所有key，必须在1个slot上，否则直接返回error, "-ERR eval/evalsha command keys must in same slot\\r\\n"

### monitor ###

必要情况下使用monitor命令时，慎用

### **Jedis参数配置** ###

<table> 
 <thead> 
  <tr> 
   <th>参数名</th> 
   <th>含义</th> 
   <th>默认值</th> 
   <th style="width:281px;">使用建议</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>maxTotal</td> 
   <td>资源池中最大连接数</td> 
   <td>8</td> 
   <td style="width:281px;"> <p>最大连接数</p> <p>这个值不是越大越好，一方面连接太多占用客户端和服务端资源，另一方面对于Redis这种高QPS的服务器，一个大命令的阻塞即使设置再大资源池仍然会无济于事。</p> <p>1.一次命令时间（borrow|return resource + Jedis执行命令(含网络) ）的平均耗时约为1ms，一个连接的QPS大约是1000;</p> <p>2.业务期望的QPS是50000</p> <p>那么理论上需要的资源池大小是50000 / 1000 = 50个</p> </td> 
  </tr> 
  <tr> 
   <td>maxIdle</td> 
   <td>资源池允许最大空闲的连接数</td> 
   <td>8</td> 
   <td style="width:281px;"> <p>实际上才是业务需要的最大连接数，而maxTotal是为了给出余量，所以maxIdle不要设置过小，否则会有new Jedis(新连接)开销。</p> <p>连接池的最佳性能是maxTotal = maxIdle ,这样就避免连接池伸缩带来的性能干扰。但是如果并发量不大或者maxTotal设置过高，会导致不必要的连接资源浪费。</p> </td> 
  </tr> 
  <tr> 
   <td>minIdle</td> 
   <td>资源池确保最少空闲的连接数</td> 
   <td>0</td> 
   <td style="width:281px;">minIdle是为了控制空闲资源监测</td> 
  </tr> 
  <tr> 
   <td>blockWhenExhausted</td> 
   <td>当资源池用尽后，调用者是否要等待。只有当为true时，下面的maxWaitMillis才会生效</td> 
   <td>true</td> 
   <td style="width:281px;">建议使用默认值</td> 
  </tr> 
  <tr> 
   <td>maxWaitMillis</td> 
   <td>当资源池连接用尽后，调用者的最大等待时间(单位为毫秒)</td> 
   <td>-1：表示永不超时</td> 
   <td style="width:281px;">不建议使用默认值</td> 
  </tr> 
  <tr> 
   <td>testOnBorrow</td> 
   <td>向资源池借用连接时是否做连接有效性检测(ping)，无效连接会被移除</td> 
   <td>false</td> 
   <td style="width:281px;">业务量很大时候建议设置为false(多一次ping的开销)。</td> 
  </tr> 
  <tr> 
   <td>testOnReturn</td> 
   <td>向资源池归还连接时是否做连接有效性检测(ping)，无效连接会被移除</td> 
   <td>false</td> 
   <td style="width:281px;">业务量很大时候建议设置为false(多一次ping的开销)。</td> 
  </tr> 
  <tr> 
   <td>jmxEnabled</td> 
   <td>是否开启jmx监控，可用于监控</td> 
   <td>true</td> 
   <td style="width:281px;">建议开启，但应用本身也要开启</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <thead> 
  <tr> 
   <th>参数名</th> 
   <th>含义</th> 
   <th>默认值</th> 
   <th style="width:332px;">使用建议</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>testWhileIdle</td> 
   <td>是否开启空闲资源监测</td> 
   <td>false</td> 
   <td style="width:332px;">true</td> 
  </tr> 
  <tr> 
   <td>timeBetweenEvictionRunsMillis</td> 
   <td>空闲资源的检测周期(单位为毫秒)</td> 
   <td>-1：不检测</td> 
   <td style="width:332px;">建议设置，周期自行选择，也可以默认也可以使用下面JedisPoolConfig中的配置</td> 
  </tr> 
  <tr> 
   <td>minEvictableIdleTimeMillis</td> 
   <td>资源池中资源最小空闲时间(单位为毫秒)，达到此值后空闲资源将被移除</td> 
   <td>1000<em>60&nbsp;</em>30 = 30分钟</td> 
   <td style="width:332px;">可根据自身业务决定，大部分默认值即可，也可以考虑使用下面JeidsPoolConfig中的配置</td> 
  </tr> 
  <tr> 
   <td>numTestsPerEvictionRun</td> 
   <td>做空闲资源检测时，每次的采样数</td> 
   <td>3</td> 
   <td style="width:332px;">可根据自身应用连接数进行微调,如果设置为-1，就是对所有连接做空闲监测</td> 
  </tr> 
 </tbody> 
</table>

### **Jedis常见异常** ###

*  [一、redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool][redis.clients.jedis.exceptions.JedisConnectionException_ Could not get a resource from the pool]
 *  [二、redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream][redis.clients.jedis.exceptions.JedisConnectionException_ Unexpected end of stream]
 *  [三、redis.clients.jedis.exceptions.JedisDataException: ERR illegal address][redis.clients.jedis.exceptions.JedisDataException_ ERR illegal address]
 *  [四、redis.clients.jedis.exceptions.JedisDataException: ERR max number of clients reached][redis.clients.jedis.exceptions.JedisDataException_ ERR max number of clients reached]
 *  [五、redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out][redis.clients.jedis.exceptions.JedisConnectionException_ java.net.SocketTimeoutException_ Read timed out]
 *  [六、redis.clients.jedis.exceptions.JedisDataException: NOAUTH Authentication required][redis.clients.jedis.exceptions.JedisDataException_ NOAUTH Authentication required]
 *  [七、redis.clients.jedis.exceptions.JedisDataException: EXECABORT Transaction discarded because of previous errors][redis.clients.jedis.exceptions.JedisDataException_ EXECABORT Transaction discarded because of previous errors]
 *  [八、java.lang.ClassCastException: java.lang.Long cannot be cast to java.util.List][java.lang.ClassCastException_ java.lang.Long cannot be cast to java.util.List]
 *  [九、redis.clients.jedis.exceptions.JedisDataException: WRONGTYPE Operation against a key holding the wrong kind of value][redis.clients.jedis.exceptions.JedisDataException_ WRONGTYPE Operation against a key holding the wrong kind of value]
 *  [十、redis.clients.jedis.exceptions.JedisDataException: OOM command not allowed when used memory > 'maxmemory'][redis.clients.jedis.exceptions.JedisDataException_ OOM command not allowed when used memory _ _maxmemory]
 *  [十一、redis.clients.jedis.exceptions.JedisDataException: LOADING Redis is loading the dataset in memory][redis.clients.jedis.exceptions.JedisDataException_ LOADING Redis is loading the dataset in memory]
 *  [十二、redis.clients.jedis.exceptions.JedisDataException: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.][redis.clients.jedis.exceptions.JedisDataException_ BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.]
 *  [十三、redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out][redis.clients.jedis.exceptions.JedisConnectionException_ java.net.SocketTimeoutException_ connect timed out]
 *  [十四、UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.][UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.]
 *  [十五、java.lang.NoClassDefFoundError][java.lang.NoClassDefFoundError]
 *  [十六、redis.clients.jedis.exceptions.JedisDataException: ERR unknown command][redis.clients.jedis.exceptions.JedisDataException_ ERR unknown command]
 *  [十七、redis.clients.jedis.exceptions.JedisDataException: Please close pipeline or multi block before calling this method.][redis.clients.jedis.exceptions.JedisDataException_ Please close pipeline or multi block before calling this method.]
 *  [十八、redis.clients.jedis.exceptions.JedisDataException: ERR command role not support for normal user.][redis.clients.jedis.exceptions.JedisDataException_ ERR command role not support for normal user.]

持续更新

[Redis]: https://blog.csdn.net/alex_xfboy/article/details/88983770
[90897223]: /images/20220122/1120e734fe514eddba7b2799cde3055c.png
[redis.clients.jedis.exceptions.JedisConnectionException_ Could not get a resource from the pool]: https://yq.aliyun.com/articles/236384#cc1
[redis.clients.jedis.exceptions.JedisConnectionException_ Unexpected end of stream]: https://yq.aliyun.com/articles/236384#cc2
[redis.clients.jedis.exceptions.JedisDataException_ ERR illegal address]: https://yq.aliyun.com/articles/236384#cc3
[redis.clients.jedis.exceptions.JedisDataException_ ERR max number of clients reached]: https://yq.aliyun.com/articles/236384#cc4
[redis.clients.jedis.exceptions.JedisConnectionException_ java.net.SocketTimeoutException_ Read timed out]: https://yq.aliyun.com/articles/236384#cc5
[redis.clients.jedis.exceptions.JedisDataException_ NOAUTH Authentication required]: https://yq.aliyun.com/articles/236384#cc6
[redis.clients.jedis.exceptions.JedisDataException_ EXECABORT Transaction discarded because of previous errors]: https://yq.aliyun.com/articles/236384#cc7
[java.lang.ClassCastException_ java.lang.Long cannot be cast to java.util.List]: https://yq.aliyun.com/articles/236384#cc8
[redis.clients.jedis.exceptions.JedisDataException_ WRONGTYPE Operation against a key holding the wrong kind of value]: https://yq.aliyun.com/articles/236384#cc9
[redis.clients.jedis.exceptions.JedisDataException_ OOM command not allowed when used memory _ _maxmemory]: https://yq.aliyun.com/articles/236384#cc10
[redis.clients.jedis.exceptions.JedisDataException_ LOADING Redis is loading the dataset in memory]: https://yq.aliyun.com/articles/236384#cc11
[redis.clients.jedis.exceptions.JedisDataException_ BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.]: https://yq.aliyun.com/articles/236384#cc12
[redis.clients.jedis.exceptions.JedisConnectionException_ java.net.SocketTimeoutException_ connect timed out]: https://yq.aliyun.com/articles/236384#cc13
[UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.]: https://yq.aliyun.com/articles/236384#cc14
[java.lang.NoClassDefFoundError]: https://yq.aliyun.com/articles/236384#cc15
[redis.clients.jedis.exceptions.JedisDataException_ ERR unknown command]: https://yq.aliyun.com/articles/236384#cc16
[redis.clients.jedis.exceptions.JedisDataException_ Please close pipeline or multi block before calling this method.]: https://yq.aliyun.com/articles/236384#cc17
[redis.clients.jedis.exceptions.JedisDataException_ ERR command role not support for normal user.]: https://yq.aliyun.com/articles/236384#cc18