Intel® Extension for Transformers 量化文本嵌入
載入由 Intel® Extension for Transformers (ITREX) 產生的量化 BGE 嵌入模型，並使用 ITREX Neural Engine (高效能 NLP 後端) 加速模型推論，且不犧牲準確性。
請參閱我們的部落格文章 Efficient Natural Language Embedding Models with Intel Extension for Transformers 與 BGE 優化範例 以取得更多詳細資訊。
from langchain_community.embeddings import QuantizedBgeEmbeddings
model_name = "Intel/bge-small-en-v1.5-sts-int8-static-inc"
encode_kwargs = {"normalize_embeddings": True} # set True to compute cosine similarity
model = QuantizedBgeEmbeddings(
model_name=model_name,
encode_kwargs=encode_kwargs,
query_instruction="Represent this sentence for searching relevant passages: ",
)
API 參考：QuantizedBgeEmbeddings
/home/yuwenzho/.conda/envs/bge/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
2024-03-04 10:17:17 [INFO] Start to extarct onnx model ops...
2024-03-04 10:17:17 [INFO] Extract onnxruntime model done...
2024-03-04 10:17:17 [INFO] Start to implement Sub-Graph matching and replacing...
2024-03-04 10:17:18 [INFO] Sub-Graph match and replace done...
使用方式
text = "This is a test document."
query_result = model.embed_query(text)
doc_result = model.embed_documents([text])