MongoDB 固定集合（Capped Collections）

心已赠人 2023-10-12 14:13 40阅读 0赞

## ![b60911e2f39c42a1ae22ff514325a0b4.png][] ##

## ![813ef536f8f34a92b231df4436da93d8.png][] ##

## （English Version）Capped Collections ##

### Overview ###

[Capped collections][] are fixed-size collections that support high-throughput operations that insert and retrieve documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.

See [createCollection()][createCollection] or [create][] for more information on creating capped collections.

### Behavior ###

#### Insertion Order ####

Capped collections guarantee preservation of the insertion order. As a result, queries do not need an index to return documents in insertion order. Without this indexing overhead, capped collections can support higher insertion throughput.

#### Automatic Removal of Oldest Documents ####

To make room for new documents, capped collections automatically remove the oldest documents in the collection without requiring scripts or explicit remove operations.

Consider the following potential use cases for capped collections:

*  Store log information generated by high-volume systems. Inserting documents in a capped collection without an index is close to the speed of writing log information directly to a file system. Furthermore, the built-in *first-in-first-out* property maintains the order of events, while managing storage use.
 *  Cache small amounts of data in a capped collections. Since caches are read rather than write heavy, you would either need to ensure that this collection *always* remains in the working set (i.e. in RAM) *or* accept some write penalty for the required index or indexes.

For example, the [oplog.rs][] collection that stores a log of the operations in a [replica set][] uses a capped collection. Starting in MongoDB 4.0, unlike other capped collections, the oplog can grow past its configured size limit to avoid deleting the [majority commit point.][]

#### `_id` Index ####

Capped collections have an `_id` field and an index on the `_id` field by default.

### Restrictions and Recommendations ###

#### Reads ####

Starting in MongoDB 5.0, you cannot use read concern ["snapshot"][snapshot] when reading from a [capped][] collection.

#### Updates ####

If you plan to update documents in a capped collection, create an index so that these update operations do not require a collection scan.

#### Sharding ####

You cannot shard a capped collection.

#### Query Efficiency ####

Use natural ordering to retrieve the most recently inserted elements from the collection efficiently. This is similar to using the `tail` command on a log file.

#### Aggregation `$out` ####

The aggregation pipeline stage [$out][out] cannot write results to a capped collection.

#### Transactions ####

Starting in MongoDB 4.2, you cannot write to [capped][] collections in [transactions.][]

#### Stable API ####

Capped collections are not supported in [Stable API][] V1.

### Procedures ###

#### Create a Capped Collection ####

You must create capped collections explicitly using the [db.createCollection()][createCollection] method, which is a [mongosh][] helper for the [create][] command. When creating a capped collection you must specify the maximum size of the collection in bytes, which MongoDB will pre-allocate for the collection. The size of the capped collection includes a small amount of space for internal overhead.

<table> 
 <tbody> 
  <tr> 
   <td><code>db.createCollection( "log", { capped: true, size: 100000 } )</code></td> 
  </tr> 
 </tbody> 
</table>

If the `size` field is less than or equal to 4096, then the collection will have a cap of 4096 bytes. Otherwise, MongoDB will raise the provided size to make it an integer multiple of 256.

Additionally, you may also specify a maximum number of documents for the collection using the `max` field as in the following document:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.createCollection("log", { capped : true, size : 5242880, max : 5000 } )</code></td> 
  </tr> 
 </tbody> 
</table>

#### Query a Capped Collection ####

If you perform a [find()][find] on a capped collection with no ordering specified, MongoDB guarantees that the ordering of results is the same as the insertion order.

To retrieve documents in reverse insertion order, issue [find()][find] along with the [sort()][sort] method with the [$natural][natural] parameter set to `-1`, as shown in the following example:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.cappedCollection.find().sort( { $natural: -1 } )</code></td> 
  </tr> 
 </tbody> 
</table>

#### Check if a Collection is Capped ####

Use the [isCapped()][isCapped] method to determine if a collection is capped, as follows:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.collection.isCapped()</code></td> 
  </tr> 
 </tbody> 
</table>

#### Convert a Collection to Capped ####

You can convert a non-capped collection to a capped collection with the [convertToCapped][] command:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.runCommand({"convertToCapped": "mycoll", size: 100000});</code></td> 
  </tr> 
 </tbody> 
</table>

The `size` parameter specifies the size of the capped collection in bytes.

This holds a database exclusive lock for the duration of the operation. Other operations which lock the same database will be blocked until the operation completes. See [What locks are taken by some common client operations?][What locks are taken by some common client operations] for operations that lock the database.

#### Change a Capped Collection's Size ####

*New in version 6.0*.

You can resize a capped collection using the [collMod][] command's `cappedSize` option to set the `cappedSize` in bytes. `cappedSize` must be greater than `0` and less than `1e+15` (1 PB).

For example, the following command sets the maximum size of the `"log"` capped collection to 100000 bytes:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.runCommand( { collMod: "log", cappedSize: 100000 } )</code></td> 
  </tr> 
 </tbody> 
</table>

#### Change the Maximum Number of Documents in a Capped Collection ####

*New in version 6.0*.

To change the maximum number of documents in a capped collection, use the [collMod][] command's `cappedMax` option. If `cappedMax` is less than or equal to `0`, there is no maximum document limit.

For example, the following command sets the maximum number of documents in the `"log"` capped collection to 500:

<table> 
 <tbody> 
  <tr> 
   <td><code>db.runCommand( { collMod: "log", cappedMax: 500 } )</code></td> 
  </tr> 
 </tbody> 
</table>

#### Tailable Cursor ####

You can use a [tailable cursor][] with capped collections. Similar to the Unix `tail -f` command, the tailable cursor "tails" the end of a capped collection. As new documents are inserted into the capped collection, you can use the tailable cursor to continue retrieving documents.

See [Tailable Cursors][] for information on creating a tailable cursor.

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* we had a big issue in the project in the past , and it caused the tailable cursor cannot be used for streaming data . so if you are using node.js streaming data , please enable the capped connection feature .

sort by [Moshow郑锴的CSDN![icon-default.png?t=N4P3][icon-default.png_t_N4P3]https://zhengkai.blog.csdn.net/][Moshow_CSDN_icon-default.png_t_N4P3_icon-default.png_t_N4P3_https_zhengkai.blog.csdn.net]

## （中文版）MongoDB 固定集合 ##

MongoDB 固定集合（Capped Collections）是性能出色且有着固定大小的集合，对于大小固定，我们可以想象其就像一个环形队列，当集合空间用完后，再插入的元素就会覆盖最初始的头部的元素！

--------------------

### 创建固定集合 ###

我们通过createCollection来创建一个固定集合，且capped选项设置为true：

>db.createCollection("cappedLogCollection",{capped:true,size:10000})

还可以指定文档个数,加上max:1000属性：

>db.createCollection("cappedLogCollection",{capped:true,size:10000,max:1000})

判断集合是否为固定集合:

>db.cappedLogCollection.isCapped()

如果需要将已存在的集合转换为固定集合可以使用以下命令：

>db.runCommand({"convertToCapped":"posts",size:10000})

以上代码将我们已存在的 posts 集合转换为固定集合。

**\*\*\*补充**

db.createCollection("cappedLogCollection",{capped:true,size:10000,max:1000})

**size** 是整个集合空间大小，单位为【字节】

**max** 是集合文档个数上线，单位是【个】

如果空间大小到达上限，则插入下一个文档时，会覆盖第一个文档；如果文档个数到达上限，同样插入下一个文档时，会覆盖第一个文档。两个参数上限判断取的是【与】的逻辑。

--------------------

### 固定集合查询 ###

固定集合文档按照插入顺序储存的,默认情况下查询就是按照插入顺序返回的,也可以使用$natural调整返回顺序。

>db.cappedLogCollection.find().sort({$natural:-1})

--------------------

### 固定集合的功能特点 ###

可以插入及更新,但更新不能超出collection的大小,否则更新失败,不允许删除,但是可以调用drop()删除集合中的所有行,但是drop后需要显式地重建集合。

在32位机子上一个cappped collection的最大值约为482.5M,64位上只受系统文件大小的限制。

--------------------

### 固定集合属性及用法 ###

#### 属性 ####

*  属性1:对固定集合进行插入速度极快
 *  属性2:按照插入顺序的查询输出速度极快
 *  属性3:能够在插入最新数据时,淘汰最早的数据

#### 用法 ####

*  用法1:储存日志信息
 *  用法2:缓存一些少量的文档

[b60911e2f39c42a1ae22ff514325a0b4.png]: https://img-blog.csdnimg.cn/b60911e2f39c42a1ae22ff514325a0b4.png
[813ef536f8f34a92b231df4436da93d8.png]: https://img-blog.csdnimg.cn/813ef536f8f34a92b231df4436da93d8.png
[Capped collections]: https://www.mongodb.com/docs/manual/reference/glossary/#std-term-capped-collection
[createCollection]: https://www.mongodb.com/docs/manual/reference/method/db.createCollection/#mongodb-method-db.createCollection
[create]: https://www.mongodb.com/docs/manual/reference/command/create/#mongodb-dbcommand-dbcmd.create
[oplog.rs]: https://www.mongodb.com/docs/manual/reference/glossary/#std-term-oplog
[replica set]: https://www.mongodb.com/docs/manual/reference/glossary/#std-term-replica-set
[majority commit point.]: https://www.mongodb.com/docs/manual/reference/command/replSetGetStatus/#mongodb-data-replSetGetStatus.optimes.lastCommittedOpTime
[snapshot]: https://www.mongodb.com/docs/manual/reference/read-concern-snapshot/#mongodb-readconcern-readconcern.-snapshot-
[capped]: https://www.mongodb.com/docs/manual/core/capped-collections/
[out]: https://www.mongodb.com/docs/manual/reference/operator/aggregation/out/#mongodb-pipeline-pipe.-out
[transactions.]: https://www.mongodb.com/docs/manual/core/transactions/
[Stable API]: https://www.mongodb.com/docs/manual/reference/stable-api/#std-label-stable-api
[mongosh]: https://www.mongodb.com/docs/mongodb-shell/#mongodb-binary-bin.mongosh
[find]: https://www.mongodb.com/docs/manual/reference/method/db.collection.find/#mongodb-method-db.collection.find
[sort]: https://www.mongodb.com/docs/manual/reference/method/cursor.sort/#mongodb-method-cursor.sort
[natural]: https://www.mongodb.com/docs/manual/reference/operator/meta/natural/#mongodb-operator-metaOp.-natural
[isCapped]: https://www.mongodb.com/docs/manual/reference/method/db.collection.isCapped/#mongodb-method-db.collection.isCapped
[convertToCapped]: https://www.mongodb.com/docs/manual/reference/command/convertToCapped/#mongodb-dbcommand-dbcmd.convertToCapped
[What locks are taken by some common client operations]: https://www.mongodb.com/docs/manual/faq/concurrency/#std-label-faq-concurrency-operations-locks
[collMod]: https://www.mongodb.com/docs/manual/reference/command/collMod/#mongodb-dbcommand-dbcmd.collMod
[tailable cursor]: https://www.mongodb.com/docs/manual/reference/glossary/#std-term-tailable-cursor
[Tailable Cursors]: https://www.mongodb.com/docs/manual/core/tailable-cursors/
[icon-default.png_t_N4P3]: https://csdnimg.cn/release/blog_editor_html/release2.3.0/ckeditor/plugins/CsdnLink/icons/icon-default.png?t=N4P3
[Moshow_CSDN_icon-default.png_t_N4P3_icon-default.png_t_N4P3_https_zhengkai.blog.csdn.net]: https://zhengkai.blog.csdn.net/