24----txtai 4.0 中的新功能

「爱情、让人受尽委屈。」 2023-09-27 19:38 102阅读 0赞

txtai 4.0 带来了许多主要的功能增强，最重要的是能够在 txtai 中存储完整的文档内容和文本。本文将通过示例涵盖所有更改。

安装依赖  
安装txtai和所有依赖项。  
pip install txtai  
内容存储  
到目前为止，使用 txtai，一旦文本被矢量化，就无法再追溯到输入文本。仅存储文档 ID 和向量。结果由 ID 和分数组成。开发人员有责任解决与外部数据存储的匹配问题。

txtai 4.0 带来了重大的范式转变。内容现在可以与嵌入向量一起存储。这为 txtai 开启了许多激动人心的可能性！

让我们看看下面的经典 txtai 示例。  
from txtai.embeddings import Embeddings

data = \[“US tops 5 million confirmed virus cases”,  
“Canada’s last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg”,  
“Beijing mobilises invasion craft along coast as Taiwan tensions escalate”,  
“The National Park Service warns against sacrificing slower friends in a bear attack”,  
“Maine man wins $1M from $25 lottery ticket”,  
“Make huge profits without work, earn up to $100,000 a day”\]

## Create embeddings index with content enabled. The default behavior is to only store indexed vectors. ##

embeddings = Embeddings(\{“path”: “sentence-transformers/nli-mpnet-base-v2”, “content”: True, “objects”: True\})

## Create an index for the list of text ##

embeddings.index(\[(uid, text, None) for uid, text in enumerate(data)\])

print(“%-20s %s” % (“Query”, “Best Match”))  
print(“-” \* 50)

## Run an embeddings search for each query ##

for query in (“feel good story”, “climate change”, “public health story”, “war”, “wildlife”, “asia”, “lucky”, “dishonest junk”):  
\# Extract text field from result  
text = embeddings.search(query, 1)\[0\]\[“text”\]

# Print text
    print("%-20s %s" % (query, text))

### Query Best Match ###

feel good story Maine man wins $1M from $25 lottery ticket  
climate change Canada’s last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg  
public health story US tops 5 million confirmed virus cases  
war Beijing mobilises invasion craft along coast as Taiwan tensions escalate  
wildlife The National Park Service warns against sacrificing slower friends in a bear attack  
asia Beijing mobilises invasion craft along coast as Taiwan tensions escalate  
lucky Maine man