Javelin AI Gateway 教學
這個 Jupyter Notebook 將會探討如何使用 Python SDK 與 Javelin AI Gateway 互動。 Javelin AI Gateway 透過提供安全且統一的端點，促進 OpenAI、Cohere、Anthropic 等大型語言模型 (LLM) 的利用。 閘道本身提供一種集中機制，以系統性地推出模型、提供存取安全性、企業的策略和成本護欄等等。
有關 Javelin 的所有功能與優勢的完整清單，請造訪 www.getjavelin.io
步驟 1：簡介
Javelin AI Gateway 是一個企業級的 AI 應用程式 API 閘道。 它整合了強大的存取安全性，確保與大型語言模型的安全互動。 請在官方文件中了解更多資訊。
步驟 2：安裝
在開始之前，我們必須安裝 javelin_sdk 並將 Javelin API 金鑰設定為環境變數。
pip install 'javelin_sdk'
Requirement already satisfied: javelin_sdk in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (0.1.8)
Requirement already satisfied: httpx<0.25.0,>=0.24.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (0.24.1)
Requirement already satisfied: pydantic<2.0.0,>=1.10.7 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (1.10.12)
Requirement already satisfied: certifi in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (2023.5.7)
Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (0.17.3)
Requirement already satisfied: idna in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (3.4)
Requirement already satisfied: sniffio in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (1.3.0)
Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from pydantic<2.0.0,>=1.10.7->javelin_sdk) (4.7.1)
Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (0.14.0)
Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (3.7.1)
Note: you may need to restart the kernel to use updated packages.
步驟 3：完成範例
本節將示範如何與 Javelin AI Gateway 互動，以從大型語言模型取得完成。 這是一個示範此操作的 Python 腳本：（注意）假設您已在閘道中設定一個名為「eng_dept03」的路由
from langchain.chains import LLMChain
from langchain_community.llms import JavelinAIGateway
from langchain_core.prompts import PromptTemplate
route_completions = "eng_dept03"
gateway = JavelinAIGateway(
gateway_uri="https://127.0.0.1:8000", # replace with service URL or host/port of Javelin
route=route_completions,
model_name="gpt-3.5-turbo-instruct",
)
prompt = PromptTemplate("Translate the following English text to French: {text}")
llmchain = LLMChain(llm=gateway, prompt=prompt)
result = llmchain.run("podcast player")
print(result)
---------------------------------------------------------------------------
``````output
ImportError Traceback (most recent call last)
``````output
Cell In[6], line 2
1 from langchain.chains import LLMChain
----> 2 from langchain.llms import JavelinAIGateway
3 from langchain.prompts import PromptTemplate
5 route_completions = "eng_dept03"
``````output
ImportError: cannot import name 'JavelinAIGateway' from 'langchain.llms' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/llms/__init__.py)
步驟 4：嵌入範例
本節示範如何使用 Javelin AI Gateway 取得文字查詢和文件的嵌入。 這是一個說明此操作的 Python 腳本：（注意）假設您已在閘道中設定一個名為「embeddings」的路由
from langchain_community.embeddings import JavelinAIGatewayEmbeddings
embeddings = JavelinAIGatewayEmbeddings(
gateway_uri="https://127.0.0.1:8000", # replace with service URL or host/port of Javelin
route="embeddings",
)
print(embeddings.embed_query("hello"))
print(embeddings.embed_documents(["hello"]))
API 參考：JavelinAIGatewayEmbeddings
---------------------------------------------------------------------------
``````output
ImportError Traceback (most recent call last)
``````output
Cell In[9], line 1
----> 1 from langchain.embeddings import JavelinAIGatewayEmbeddings
2 from langchain.embeddings.openai import OpenAIEmbeddings
4 embeddings = JavelinAIGatewayEmbeddings(
5 gateway_uri="https://127.0.0.1:8000", # replace with service URL or host/port of Javelin
6 route="embeddings",
7 )
``````output
ImportError: cannot import name 'JavelinAIGatewayEmbeddings' from 'langchain.embeddings' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/embeddings/__init__.py)
步驟 5：聊天範例
本節說明如何與 Javelin AI Gateway 互動，以促進與大型語言模型的聊天。 以下是一個 Python 腳本，演示了此過程：（注意）假設您已在閘道中設定一個名為“mychatbot_route”的路由
from langchain_community.chat_models import ChatJavelinAIGateway
from langchain_core.messages import HumanMessage, SystemMessage
messages = [
SystemMessage(
content="You are a helpful assistant that translates English to French."
),
HumanMessage(
content="Artificial Intelligence has the power to transform humanity and make the world a better place"
),
]
chat = ChatJavelinAIGateway(
gateway_uri="https://127.0.0.1:8000", # replace with service URL or host/port of Javelin
route="mychatbot_route",
model_name="gpt-3.5-turbo",
params={"temperature": 0.1},
)
print(chat(messages))
---------------------------------------------------------------------------
``````output
ImportError Traceback (most recent call last)
``````output
Cell In[8], line 1
----> 1 from langchain.chat_models import ChatJavelinAIGateway
2 from langchain.schema import HumanMessage, SystemMessage
4 messages = [
5 SystemMessage(
6 content="You are a helpful assistant that translates English to French."
(...)
10 ),
11 ]
``````output
ImportError: cannot import name 'ChatJavelinAIGateway' from 'langchain.chat_models' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/chat_models/__init__.py)
步驟 6：結論。本教學課程介紹了 Javelin AI Gateway，並演示了如何使用 Python SDK 與其互動。 請記住查看 Javelin Python SDK 以獲取更多範例，並瀏覽官方文件以獲取更多詳細資訊。
祝您編碼愉快！