Jaguar Vector Database

1. 它是一個分散式向量資料庫 (It is a distributed vector database)
2. JaguarDB 的 “ZeroMove” 功能可實現即時水平擴展 (The “ZeroMove” feature of JaguarDB enables instant horizontal scalability)
3. 多模態：嵌入、文字、圖像、影片、PDF、音訊、時間序列和地理空間 (Multimodal: embeddings, text, images, videos, PDFs, audio, time series, and geospatial)
4. 所有主節點：允許並行讀取和寫入 (All-masters: allows both parallel reads and writes)
5. 異常偵測功能 (Anomaly detection capabilities)
6. RAG 支援：將 LLM 與專有和即時資料結合 (RAG support: combines LLM with proprietary and real-time data)
7. 共享元數據：跨多個向量索引共享元數據 (Shared metadata: sharing of metadata across multiple vector indexes)
8. 距離度量：歐幾里得、餘弦、內積、曼哈頓、切比雪夫、漢明、傑卡德、閔可夫斯基 (Distance metrics: Euclidean, Cosine, InnerProduct, Manhatten, Chebyshev, Hamming, Jeccard, Minkowski)

先決條件

執行此檔案中的範例有兩個要求。(There are two requirements for running the examples in this file.)

1. 您必須安裝並設定 JaguarDB 伺服器及其 HTTP 閘道伺服器。 請參閱以下說明：www.jaguardb.com 如需在 docker 環境中快速設定：docker pull jaguardb/jaguardb_with_http docker run -d -p 8888:8888 -p 8080:8080 --name jaguardb_with_http jaguardb/jaguardb_with_http (You must install and set up the JaguarDB server and its HTTP gateway server. Please refer to the instructions in: www.jaguardb.com For quick setup in docker environment: docker pull jaguardb/jaguardb_with_http docker run -d -p 8888:8888 -p 8080:8080 --name jaguardb_with_http jaguardb/jaguardb_with_http)

2. 您必須安裝 JaguarDB 的 http 客戶端套件 (You must install the http client package for JaguarDB)

       pip install -U jaguardb-http-client

3. 您需要使用 pip install -qU langchain-community 安裝 langchain-community 才能使用此整合 (You'll need to install langchain-community with pip install -qU langchain-community to use this integration)

使用 Langchain 的 RAG

本節示範如何在 langchain 軟體堆疊中與 LLM 和 Jaguar 一起聊天。(This section demonstrates chatting with LLM together with Jaguar in the langchain software stack.)

from langchain.chains import RetrievalQAWithSourcesChain
from langchain_community.document_loaders import TextLoader
from langchain_community.vectorstores.jaguar import Jaguar
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI, OpenAI, OpenAIEmbeddings
from langchain_text_splitters import CharacterTextSplitter

""" 
Load a text file into a set of documents 
"""
loader = TextLoader("../../how_to/state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=300)
docs = text_splitter.split_documents(documents)

"""
Instantiate a Jaguar vector store
"""
### Jaguar HTTP endpoint
url = "http://192.168.5.88:8080/fwww/"

### Use OpenAI embedding model
embeddings = OpenAIEmbeddings()

### Pod is a database for vectors
pod = "vdb"

### Vector store name
store = "langchain_rag_store"

### Vector index name
vector_index = "v"

### Type of the vector index
# cosine: distance metric
# fraction: embedding vectors are decimal numbers
# float: values stored with floating-point numbers
vector_type = "cosine_fraction_float"

### Dimension of each embedding vector
vector_dimension = 1536

### Instantiate a Jaguar store object
vectorstore = Jaguar(
    pod, store, vector_index, vector_type, vector_dimension, url, embeddings
)

"""
Login must be performed to authorize the client.
The environment variable JAGUAR_API_KEY or file $HOME/.jagrc
should contain the API key for accessing JaguarDB servers.
"""
vectorstore.login()


"""
Create vector store on the JaguarDB database server.
This should be done only once.
"""
# Extra metadata fields for the vector store
metadata = "category char(16)"

# Number of characters for the text field of the store
text_size = 4096

#  Create a vector store on the server
vectorstore.create(metadata, text_size)

"""
Add the texts from the text splitter to our vectorstore
"""
vectorstore.add_documents(docs)
# or tag the documents:
# vectorstore.add_documents(more_docs, text_tag="tags to these documents")

""" Get the retriever object """
retriever = vectorstore.as_retriever()
# retriever = vectorstore.as_retriever(search_kwargs={"where": "m1='123' and m2='abc'"})

template = """You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.
Question: {question}
Context: {context}
Answer:
"""
prompt = ChatPromptTemplate.from_template(template)

""" Obtain a Large Language Model """
LLM = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

""" Create a chain for the RAG flow """
rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | LLM
    | StrOutputParser()
)

resp = rag_chain.invoke("What did the president say about Justice Breyer?")
print(resp)

API 參考：RetrievalQAWithSourcesChain | TextLoader | Jaguar | StrOutputParser | ChatPromptTemplate | RunnablePassthrough | ChatOpenAI | OpenAI | OpenAIEmbeddings | CharacterTextSplitter

與 Jaguar Vector Store 互動

使用者可以直接與 Jaguar 向量儲存互動，以進行相似性搜尋和異常偵測。(Users can interact directly with the Jaguar vector store for similarity search and anomaly detection.)

from langchain_community.vectorstores.jaguar import Jaguar
from langchain_openai import OpenAIEmbeddings

# Instantiate a Jaguar vector store object
url = "http://192.168.3.88:8080/fwww/"
pod = "vdb"
store = "langchain_test_store"
vector_index = "v"
vector_type = "cosine_fraction_float"
vector_dimension = 10
embeddings = OpenAIEmbeddings()
vectorstore = Jaguar(
    pod, store, vector_index, vector_type, vector_dimension, url, embeddings
)

# Login for authorization
vectorstore.login()

# Create the vector store with two metadata fields
# This needs to be run only once.
metadata_str = "author char(32), category char(16)"
vectorstore.create(metadata_str, 1024)

# Add a list of texts
texts = ["foo", "bar", "baz"]
metadatas = [
    {"author": "Adam", "category": "Music"},
    {"author": "Eve", "category": "Music"},
    {"author": "John", "category": "History"},
]
ids = vectorstore.add_texts(texts=texts, metadatas=metadatas)

#  Search similar text
output = vectorstore.similarity_search(
    query="foo",
    k=1,
    metadatas=["author", "category"],
)
assert output[0].page_content == "foo"
assert output[0].metadata["author"] == "Adam"
assert output[0].metadata["category"] == "Music"
assert len(output) == 1

# Search with filtering (where)
where = "author='Eve'"
output = vectorstore.similarity_search(
    query="foo",
    k=3,
    fetch_k=9,
    where=where,
    metadatas=["author", "category"],
)
assert output[0].page_content == "bar"
assert output[0].metadata["author"] == "Eve"
assert output[0].metadata["category"] == "Music"
assert len(output) == 1

# Anomaly detection
result = vectorstore.is_anomalous(
    query="dogs can jump high",
)
assert result is False

# Remove all data in the store
vectorstore.clear()
assert vectorstore.count() == 0

# Remove the store completely
vectorstore.drop()

# Logout
vectorstore.logout()

API 參考：Jaguar | OpenAIEmbeddings

相關 (Related)

• 向量儲存 概念指南 (Vector store conceptual guide)
• 向量儲存 操作指南 (Vector store how-to guides)