DeepInfra

DeepInfra 是一種無伺服器推論即服務，可讓您存取各種 LLM 和嵌入模型。本筆記本將說明如何將 LangChain 與 DeepInfra 搭配使用於語言模型。

設定環境 API 金鑰

請務必從 DeepInfra 取得您的 API 金鑰。您必須登入並取得新的權杖。

您將獲得 1 小時的免費無伺服器 GPU 計算時間，以測試不同的模型。（請參閱此處）您可以使用 deepctl auth token 列印您的權杖

# get a new token: https://deepinfra.com/login?from=%2Fdash

from getpass import getpass

DEEPINFRA_API_TOKEN = getpass()

 ········

import os

os.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKEN

建立 DeepInfra 執行個體

您也可以使用我們的開源deepctl 工具來管理您的模型部署。您可以在此處檢視可用參數的清單。

from langchain_community.llms import DeepInfra

llm = DeepInfra(model_id="meta-llama/Llama-2-70b-chat-hf")
llm.model_kwargs = {
    "temperature": 0.7,
    "repetition_penalty": 1.2,
    "max_new_tokens": 250,
    "top_p": 0.9,
}

API 參考：DeepInfra

# run inferences directly via wrapper
llm("Who let the dogs out?")

'This is a question that has puzzled many people'

# run streaming inference
for chunk in llm.stream("Who let the dogs out?"):
    print(chunk)

 Will
 Smith
.

建立提示範本

我們將為問題與解答建立提示範本。

from langchain_core.prompts import PromptTemplate

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate.from_template(template)

API 參考：PromptTemplate

啟動 LLMChain

from langchain.chains import LLMChain

llm_chain = LLMChain(prompt=prompt, llm=llm)

API 參考：LLMChain

執行 LLMChain

提供問題並執行 LLMChain。

question = "Can penguins reach the North pole?"

llm_chain.run(question)

"Penguins are found in Antarctica and the surrounding islands, which are located at the southernmost tip of the planet. The North Pole is located at the northernmost tip of the planet, and it would be a long journey for penguins to get there. In fact, penguins don't have the ability to fly or migrate over such long distances. So, no, penguins cannot reach the North Pole. "

相關連結

• LLM 概念指南
• LLM 操作指南