Infinity
Infinity 允許使用 MIT 授權的嵌入伺服器來建立 Embeddings。
本筆記本介紹如何將 Langchain 與 Infinity Github 專案 的 Embeddings 搭配使用。
匯入
from langchain_community.embeddings import InfinityEmbeddings, InfinityEmbeddingsLocal
選項 1：從 Python 使用 infinity
可選：安裝 infinity
要安裝 infinity，請使用以下命令。 如需更多詳細資訊，請查看 Github 上的文件。 安裝 torch 和 onnx 相依性。
pip install infinity_emb[torch,optimum]
documents = [
"Baguette is a dish.",
"Paris is the capital of France.",
"numpy is a lib for linear algebra",
"You escaped what I've escaped - You'd be in Paris getting fucked up too",
]
query = "Where is Paris?"
embeddings = InfinityEmbeddingsLocal(
model="sentence-transformers/all-MiniLM-L6-v2",
# revision
revision=None,
# best to keep at 32
batch_size=32,
# for AMD/Nvidia GPUs via torch
device="cuda",
# warm up model before execution
)
async def embed():
# TODO: This function is just to showcase that your call can run async.
# important: use engine inside of `async with` statement to start/stop the batching engine.
async with embeddings:
# avoid closing and starting the engine often.
# rather keep it running.
# you may call `await embeddings.__aenter__()` and `__aexit__()
# if you are sure when to manually start/stop execution` in a more granular way
documents_embedded = await embeddings.aembed_documents(documents)
query_result = await embeddings.aembed_query(query)
print("embeddings created successful")
return documents_embedded, query_result
/home/michael/langchain/libs/langchain/.venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
/home/michael/langchain/libs/langchain/.venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
# run the async code however you would like
# if you are in a jupyter notebook, you can use the following
documents_embedded, query_result = await embed()
# (demo) compute similarity
import numpy as np
scores = np.array(documents_embedded) @ np.array(query_result).T
dict(zip(documents, scores))
選項 2：執行伺服器，並透過 API 連接
可選：請務必啟動 Infinity 實例
要安裝 infinity，請使用以下命令。 如需更多詳細資訊，請查看 Github 上的文件。
pip install infinity_emb[all]
安裝 infinity 套件
%pip install --upgrade --quiet infinity_emb[all]
啟動伺服器 - 最好從單獨的終端機完成，而不是在 Jupyter Notebook 內
model=sentence-transformers/all-MiniLM-L6-v2
port=7797
infinity_emb --port $port --model-name-or-path $model
或者，您也可以直接使用 docker
model=sentence-transformers/all-MiniLM-L6-v2
port=7797
docker run -it --gpus all -p $port:$port michaelf34/infinity:latest --model-name-or-path $model --port $port
使用您的 Infinity 實例嵌入您的文件
documents = [
"Baguette is a dish.",
"Paris is the capital of France.",
"numpy is a lib for linear algebra",
"You escaped what I've escaped - You'd be in Paris getting fucked up too",
]
query = "Where is Paris?"
#
infinity_api_url = "https://127.0.0.1:7797/v1"
# model is currently not validated.
embeddings = InfinityEmbeddings(
model="sentence-transformers/all-MiniLM-L6-v2", infinity_api_url=infinity_api_url
)
try:
documents_embedded = embeddings.embed_documents(documents)
query_result = embeddings.embed_query(query)
print("embeddings created successful")
except Exception as ex:
print(
"Make sure the infinity instance is running. Verify by clicking on "
f"{infinity_api_url.replace('v1','docs')} Exception: {ex}. "
)
Make sure the infinity instance is running. Verify by clicking on https://127.0.0.1:7797/docs Exception: HTTPConnectionPool(host='localhost', port=7797): Max retries exceeded with url: /v1/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f91c35dbd30>: Failed to establish a new connection: [Errno 111] Connection refused')).
# (demo) compute similarity
import numpy as np
scores = np.array(documents_embedded) @ np.array(query_result).T
dict(zip(documents, scores))
{'Baguette is a dish.': 0.31344215908661155,
'Paris is the capital of France.': 0.8148670296896388,
'numpy is a lib for linear algebra': 0.004429399861302009,
"You escaped what I've escaped - You'd be in Paris getting fucked up too": 0.5088476180154582}