distributed key-value store 杂思

深碍√TFBOYSˉ_ 2022-07-12 07:57 127阅读 0赞

转：[http://www.yeeach.com/tag/mongodb/][http_www.yeeach.com_tag_mongodb]

distributed key-value  store是当下比较流行的话题，尤其在构建诸如搜索引擎、IM、P2P、游戏服务器、SNS等大型互联网应用以及提供云计算服务的时候，怎样保证系统在海量数据环境下的高性能、高可靠性、高扩展性、高可用性、低成本成为所有系统架构们挖苦心思考虑的重点，而怎样解决数据库服务器的性能瓶颈是最大的挑战。

按照分布式领域的CAP理论（Consistency、Availability、Tolerance to network Partitions这三部分在任何系统架构实现时只可能同时满足其中二点，没法三者兼顾）来衡量，传统的关系数据库的ACID只满足了Consistency、Availability，因此在Partition tolerance上就很难做得好。另外传统的关系数据库处理海量数据、分布式架构时候在Performance、Scalability、Availability等方面也存在很大的局限性。

而key-value store更加注重对海量数据存取的性能、分布式、扩展性支持上，并不需要传统关系数据库的一些特征，例如：Schema、事务、完整SQL查询支持等等，因此在分布式环境下的性能相对于传统的关系数据库有较大的提升。当然不同的key-value store根据应用需求的不同，在设计理念并不完全相同，例如以Google BigTable、Amazon Dynamo为例：

BigTable is a CA system; it is strongly consistent and highly available, but can be unavailable under network partitions.  BigTable has no replication at the database level, rather replication is handled underneath by GFS.

Dynamo is an AP system; it is highly available, even under network partitions, but eventually consistent.  Data is replicated within a single cluster, so even under partitions most data is available, however one node’s latest version might not match that of another, so every reader is only guaranteed to see every write eventually.

[![distributed key-value store 杂思 - horse - Horse][distributed key-value store _ - horse - Horse]][distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse]

其实使用key-value store的历史已经很久了，以前在做电信计费时候的Radius服务器就采用Berkeley DB这样经典的key-value store作为用户数据库，只不过早期使用这样的key-value store关注的重点主要在性能上，对于分布式、海量数据的处理并非考虑的重点。

1、key-value store与RDBMS的区别

摘自:Is the Relational Database Doomed?

[![distributed key-value store 杂思 - horse - Horse][distributed key-value store _ - horse - Horse 1]][distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 1]

[![distributed key-value store 杂思 - horse - Horse][distributed key-value store _ - horse - Horse 2]][distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 2]

[![distributed key-value store 杂思 - horse - Horse][distributed key-value store _ - horse - Horse 3]][distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 3]

2、一些常用的key-value的开源项目

这些项目的设计思路大多是参考Google BigTable、Amazon Dynamo

[![distributed key-value store 杂思 - horse - Horse][distributed key-value store _ - horse - Horse 4]][distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 4]

除了以上罗列的产品外，还有如下一些产品

Tokyo cabinet

Redis

MongoDB

Oracle Berkeley DB

LightCloud

Lux IO

Drizzle

Flare

其中Tokyo cabinet、Redis、MongoDB各有特色，可以作为项目使用key-value store时候的重点评估对象。

3、与工作相关的一些可以使用key-value store的应用场合

Cache：Memcachedb、Tokyo Tyrant

网络通信：做IM、P2P时候做单点登录服务器、状态服务器；消息队列服务器（Tokyo Tyrant）

交易系统：计费系统cdr及预处理后的账单数据；交易系统风控系统（内存数据库,redis）

日志：服务器访问日志，用于web analytics使用；呼叫中心的呼叫数据；

爬虫及搜索引擎：爬取页面日志；爬取任务；索引库

SNS社区及CMS站点：可以大规模应用Tokyo Cabinet+Tokyo Tyrant，结合Solr/Lucene或这样的Sphinx搜索引擎搞定查询问题

[http_www.yeeach.com_tag_mongodb]: http://www.yeeach.com/tag/mongodb/
[distributed key-value store _ - horse - Horse]: http://www.yeeach.com/wp-content/uploads/2009/09/cap-thumb.png
[distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse]: http://www.yeeach.com/wp-content/uploads/2009/09/cap.png
[distributed key-value store _ - horse - Horse 1]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09c-thumb.png
[distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 1]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09c.png
[distributed key-value store _ - horse - Horse 2]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09e-thumb.png
[distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 2]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09e.png
[distributed key-value store _ - horse - Horse 3]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09f-thumb.png
[distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 3]: http://www.yeeach.com/wp-content/uploads/2009/09/relational-database-feb09f.png
[distributed key-value store _ - horse - Horse 4]: http://www.yeeach.com/wp-content/uploads/2009/09/richardjones-antirdbmsalistofdistributedkeyvaluestores-thumb.png
[distributed key-value store _ - horse - Horse_distributed key-value store _ - horse - Horse 4]: http://www.yeeach.com/wp-content/uploads/2009/09/richardjones-antirdbmsalistofdistributedkeyvaluestores.png