跳至主要內容

Milvus

Milvus 是一個資料庫，用於儲存、索引和管理由深度神經網路和其他機器學習 (ML) 模型產生的大量嵌入向量。

本筆記本展示如何使用與 Milvus 向量資料庫相關的功能。

Setup（設定）​

您需要使用 pip install -qU langchain-milvus 安裝 langchain-milvus 才能使用此整合。

%pip install -qU  langchain_milvus

最新版本的 pymilvus 附帶一個本地向量資料庫 Milvus Lite，非常適合原型設計。如果您有大量資料（例如超過一百萬個文檔），我們建議在 docker 或 kubernetes 上設定效能更高的 Milvus 伺服器。

Credentials（憑證）​

使用 Milvus 向量資料庫不需要憑證。

Initialization（初始化）​

pip install -qU langchain-openai
import getpass
import os

if not os.environ.get("OPENAI_API_KEY"):
  os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter API key for OpenAI: ")

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
from langchain_milvus import Milvus

# The easiest way is to use Milvus Lite where everything is stored in a local file.
# If you have a Milvus server you can use the server URI such as "https://127.0.0.1:19530".
URI = "./milvus_example.db"

vector_store = Milvus(
    embedding_function=embeddings,
    connection_args={"uri": URI},
)
API Reference:Milvus

Compartmentalize the data with Milvus Collections（使用 Milvus 集合來區隔資料）​

您可以將不同的無關文件儲存在同一個 Milvus 實例中的不同集合中，以維持上下文

以下是如何建立新集合

from langchain_core.documents import Document

vector_store_saved = Milvus.from_documents(
    [Document(page_content="foo!")],
    embeddings,
    collection_name="langchain_example",
    connection_args={"uri": URI},
)
API Reference:Document

以下是如何檢索儲存的集合

vector_store_loaded = Milvus(
    embeddings,
    connection_args={"uri": URI},
    collection_name="langchain_example",
)

Manage vector store（管理向量資料庫）​

建立向量資料庫後，我們可以透過新增和刪除不同的項目與之互動。

Add items to vector store（將項目新增至向量資料庫）​

我們可以使用 add_documents 函數將項目新增至向量資料庫。

from uuid import uuid4

from langchain_core.documents import Document

document_1 = Document(
    page_content="I had chocalate chip pancakes and scrambled eggs for breakfast this morning.",
    metadata={"source": "tweet"},
)

document_2 = Document(
    page_content="The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.",
    metadata={"source": "news"},
)

document_3 = Document(
    page_content="Building an exciting new project with LangChain - come check it out!",
    metadata={"source": "tweet"},
)

document_4 = Document(
    page_content="Robbers broke into the city bank and stole $1 million in cash.",
    metadata={"source": "news"},
)

document_5 = Document(
    page_content="Wow! That was an amazing movie. I can't wait to see it again.",
    metadata={"source": "tweet"},
)

document_6 = Document(
    page_content="Is the new iPhone worth the price? Read this review to find out.",
    metadata={"source": "website"},
)

document_7 = Document(
    page_content="The top 10 soccer players in the world right now.",
    metadata={"source": "website"},
)

document_8 = Document(
    page_content="LangGraph is the best framework for building stateful, agentic applications!",
    metadata={"source": "tweet"},
)

document_9 = Document(
    page_content="The stock market is down 500 points today due to fears of a recession.",
    metadata={"source": "news"},
)

document_10 = Document(
    page_content="I have a bad feeling I am going to get deleted :(",
    metadata={"source": "tweet"},
)

documents = [
    document_1,
    document_2,
    document_3,
    document_4,
    document_5,
    document_6,
    document_7,
    document_8,
    document_9,
    document_10,
]
uuids = [str(uuid4()) for _ in range(len(documents))]

vector_store.add_documents(documents=documents, ids=uuids)
API Reference:Document
['b0248595-2a41-4f6b-9c25-3a24c1278bb3',
 'fa642726-5329-4495-a072-187e948dd71f',
 '9905001c-a4a3-455e-ab94-72d0ed11b476',
 'eacc7256-d7fa-4036-b1f7-83d7a4bee0c5',
 '7508f7ff-c0c9-49ea-8189-634f8a0244d8',
 '2e179609-3ff7-4c6a-9e05-08978903fe26',
 'fab1f2ac-43e1-45f9-b81b-fc5d334c6508',
 '1206d237-ee3a-484f-baf2-b5ac38eeb314',
 'd43cbf9a-a772-4c40-993b-9439065fec01',
 '25e667bb-6f09-4574-a368-661069301906']

Delete items from vector store（從向量資料庫刪除項目）​

vector_store.delete(ids=[uuids[-1]])
(insert count: 0, delete count: 1, upsert count: 0, timestamp: 0, success count: 0, err count: 0, cost: 0)

Query vector store（查詢向量資料庫）​

建立向量資料庫並新增相關文件後，您很可能會希望在鏈或代理程式的執行期間查詢它。

Query directly（直接查詢）​

使用中繼資料篩選執行簡單的相似度搜尋可以如下進行

results = vector_store.similarity_search(
    "LangChain provides abstractions to make working with LLMs easy",
    k=2,
    expr='source == "tweet"',
)
for res in results:
    print(f"* {res.page_content} [{res.metadata}]")
* Building an exciting new project with LangChain - come check it out! [{'pk': '9905001c-a4a3-455e-ab94-72d0ed11b476', 'source': 'tweet'}]
* LangGraph is the best framework for building stateful, agentic applications! [{'pk': '1206d237-ee3a-484f-baf2-b5ac38eeb314', 'source': 'tweet'}]

Similarity search with score（使用分數進行相似度搜尋）​

您也可以使用分數進行搜尋

results = vector_store.similarity_search_with_score(
    "Will it be hot tomorrow?", k=1, expr='source == "news"'
)
for res, score in results:
    print(f"* [SIM={score:3f}] {res.page_content} [{res.metadata}]")
* [SIM=21192.628906] bar [{'pk': '2', 'source': 'https://example.com'}]

如需使用 Milvus 向量資料庫時可用的所有搜尋選項的完整清單，您可以瀏覽 API 參考文檔。

Query by turning into retriever（透過轉換為檢索器進行查詢）​

您也可以將向量資料庫轉換為檢索器，以便在您的鏈中更輕鬆地使用。

retriever = vector_store.as_retriever(search_type="mmr", search_kwargs={"k": 1})
retriever.invoke("Stealing from the bank is a crime", filter={"source": "news"})
[Document(metadata={'pk': 'eacc7256-d7fa-4036-b1f7-83d7a4bee0c5', 'source': 'news'}, page_content='Robbers broke into the city bank and stole $1 million in cash.')]

Usage for retrieval-augmented generation（用於檢索增強生成）​

有關如何將此向量資料庫用於檢索增強生成 (RAG) 的指南，請參閱以下各節

每位使用者的檢索​

在建立檢索應用程式時，您通常需要考慮多位使用者。這表示您可能不僅僅儲存一位使用者的資料，而是儲存多位不同使用者的資料，並且他們不應該能夠看到彼此的資料。

Milvus 建議使用 partition_key 來實作多租戶，以下是一個範例。

Partition key 的功能目前在 Milvus Lite 中尚不可用。如果您想使用它，您需要從 docker 或 kubernetes 啟動 Milvus 伺服器。

from langchain_core.documents import Document

docs = [
    Document(page_content="i worked at kensho", metadata={"namespace": "harrison"}),
    Document(page_content="i worked at facebook", metadata={"namespace": "ankush"}),
]
vectorstore = Milvus.from_documents(
    docs,
    embeddings,
    connection_args={"uri": URI},
    drop_old=True,
    partition_key_field="namespace",  # Use the "namespace" field as the partition key
)
API Reference:Document

要使用 partition key 執行搜尋，您應該在搜尋請求的布林運算式中包含以下任一項

search_kwargs={"expr": '<partition_key> == "xxxx"'}

search_kwargs={"expr": '<partition_key> == in ["xxx", "xxx"]'}

請將 <partition_key> 替換為指定為 partition key 的欄位名稱。

Milvus 會根據指定的 partition key 切換到 partition，根據 partition key 過濾實體，並在過濾後的實體中搜尋。

# This will only get documents for Ankush
vectorstore.as_retriever(search_kwargs={"expr": 'namespace == "ankush"'}).invoke(
    "where did i work?"
)
[Document(page_content='i worked at facebook', metadata={'namespace': 'ankush'})]
# This will only get documents for Harrison
vectorstore.as_retriever(search_kwargs={"expr": 'namespace == "harrison"'}).invoke(
    "where did i work?"
)
[Document(page_content='i worked at kensho', metadata={'namespace': 'harrison'})]

API 參考​

如需所有 __ModuleName__VectorStore 功能和配置的詳細文件，請前往 API 參考：https://langchain-python.dev.org.tw/api_reference/milvus/vectorstores/langchain_milvus.vectorstores.milvus.Milvus.html

這個頁面對您有幫助嗎？