Azure ML

Azure ML 是一個用於建立、訓練和部署機器學習模型的平台。使用者可以探索模型目錄中要部署的模型類型，該目錄提供來自不同提供者的基礎模型和通用模型。

本筆記本介紹了如何使用託管在 Azure ML Online Endpoint 上的 LLM。

##Installing the langchain packages needed to use the integration
%pip install -qU langchain-community

from langchain_community.llms.azureml_endpoint import AzureMLOnlineEndpoint

API 參考：AzureMLOnlineEndpoint

設定 (Set up)

您必須在 Azure ML 上部署模型或在 Azure AI studio 上部署模型，並取得以下參數

endpoint_url：端點提供的 REST 端點 URL。
endpoint_api_type：將模型部署到專用端點（託管的受管理基礎架構）時，請使用 endpoint_type='dedicated'。使用 隨用隨付 產品/服務（模型即服務）部署模型時，請使用 endpoint_type='serverless'。
endpoint_api_key：端點提供的 API 金鑰。
deployment_name：（可選）使用端點的模型部署名稱。

內容格式器 (Content Formatter)

content_formatter 參數是一個處理常式類別，用於轉換 AzureML 端點的請求和回應，以符合所需的架構。由於模型目錄中有範圍廣泛的模型，每個模型處理資料的方式可能彼此不同，因此提供了一個 ContentFormatterBase 類別，允許使用者根據自己的喜好轉換資料。提供了以下內容格式器

GPT2ContentFormatter：格式化 GPT2 的請求和回應資料
DollyContentFormatter：格式化 Dolly-v2 的請求和回應資料
HFContentFormatter：格式化文字產生 Hugging Face 模型的請求和回應資料
CustomOpenAIContentFormatter：格式化 LLaMa2 等遵循 OpenAI API 相容方案的模型請求和回應資料。

注意：OSSContentFormatter 正在被棄用，並由 GPT2ContentFormatter 取代。邏輯相同，但 GPT2ContentFormatter 是一個更合適的名稱。您仍然可以繼續使用 OSSContentFormatter，因為這些變更向後相容。

範例 (Examples)

範例：具有即時端點的 LlaMa 2 完成 (completions)

from langchain_community.llms.azureml_endpoint import (
    AzureMLEndpointApiType,
    CustomOpenAIContentFormatter,
)
from langchain_core.messages import HumanMessage

llm = AzureMLOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/score",
    endpoint_api_type=AzureMLEndpointApiType.dedicated,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIContentFormatter(),
    model_kwargs={"temperature": 0.8, "max_new_tokens": 400},
)
response = llm.invoke("Write me a song about sparkling water:")
response

API 參考：AzureMLEndpointApiType | CustomOpenAIContentFormatter | HumanMessage

模型參數也可以在調用期間指示

response = llm.invoke("Write me a song about sparkling water:", temperature=0.5)
response

範例：具有隨用隨付部署的聊天完成 (Chat completions)（模型即服務）

from langchain_community.llms.azureml_endpoint import (
    AzureMLEndpointApiType,
    CustomOpenAIContentFormatter,
)
from langchain_core.messages import HumanMessage

llm = AzureMLOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIContentFormatter(),
    model_kwargs={"temperature": 0.8, "max_new_tokens": 400},
)
response = llm.invoke("Write me a song about sparkling water:")
response

API 參考：AzureMLEndpointApiType | CustomOpenAIContentFormatter | HumanMessage

範例：自定義內容格式器 (Custom content formatter)

以下是使用來自 Hugging Face 的摘要模型的範例。

import json
import os
from typing import Dict

from langchain_community.llms.azureml_endpoint import (
    AzureMLOnlineEndpoint,
    ContentFormatterBase,
)


class CustomFormatter(ContentFormatterBase):
    content_type = "application/json"
    accepts = "application/json"

    def format_request_payload(self, prompt: str, model_kwargs: Dict) -> bytes:
        input_str = json.dumps(
            {
                "inputs": [prompt],
                "parameters": model_kwargs,
                "options": {"use_cache": False, "wait_for_model": True},
            }
        )
        return str.encode(input_str)

    def format_response_payload(self, output: bytes) -> str:
        response_json = json.loads(output)
        return response_json[0]["summary_text"]


content_formatter = CustomFormatter()

llm = AzureMLOnlineEndpoint(
    endpoint_api_type="dedicated",
    endpoint_api_key=os.getenv("BART_ENDPOINT_API_KEY"),
    endpoint_url=os.getenv("BART_ENDPOINT_URL"),
    model_kwargs={"temperature": 0.8, "max_new_tokens": 400},
    content_formatter=content_formatter,
)
large_text = """On January 7, 2020, Blockberry Creative announced that HaSeul would not participate in the promotion for Loona's 
next album because of mental health concerns. She was said to be diagnosed with "intermittent anxiety symptoms" and would be 
taking time to focus on her health.[39] On February 5, 2020, Loona released their second EP titled [#] (read as hash), along 
with the title track "So What".[40] Although HaSeul did not appear in the title track, her vocals are featured on three other 
songs on the album, including "365". Once peaked at number 1 on the daily Gaon Retail Album Chart,[41] the EP then debuted at 
number 2 on the weekly Gaon Album Chart. On March 12, 2020, Loona won their first music show trophy with "So What" on Mnet's 
M Countdown.[42]

On October 19, 2020, Loona released their third EP titled [12:00] (read as midnight),[43] accompanied by its first single 
"Why Not?". HaSeul was again not involved in the album, out of her own decision to focus on the recovery of her health.[44] 
The EP then became their first album to enter the Billboard 200, debuting at number 112.[45] On November 18, Loona released 
the music video for "Star", another song on [12:00].[46] Peaking at number 40, "Star" is Loona's first entry on the Billboard 
Mainstream Top 40, making them the second K-pop girl group to enter the chart.[47]

On June 1, 2021, Loona announced that they would be having a comeback on June 28, with their fourth EP, [&] (read as and).
[48] The following day, on June 2, a teaser was posted to Loona's official social media accounts showing twelve sets of eyes, 
confirming the return of member HaSeul who had been on hiatus since early 2020.[49] On June 12, group members YeoJin, Kim Lip, 
Choerry, and Go Won released the song "Yum-Yum" as a collaboration with Cocomong.[50] On September 8, they released another 
collaboration song named "Yummy-Yummy".[51] On June 27, 2021, Loona announced at the end of their special clip that they are 
making their Japanese debut on September 15 under Universal Music Japan sublabel EMI Records.[52] On August 27, it was announced 
that Loona will release the double A-side single, "Hula Hoop / Star Seed" on September 15, with a physical CD release on October 
20.[53] In December, Chuu filed an injunction to suspend her exclusive contract with Blockberry Creative.[54][55]
"""
summarized_text = llm.invoke(large_text)
print(summarized_text)

API 參考文檔：AzureMLOnlineEndpoint | ContentFormatterBase

範例：搭配 LLMChain 的 Dolly

from langchain.chains import LLMChain
from langchain_community.llms.azureml_endpoint import DollyContentFormatter
from langchain_core.prompts import PromptTemplate

formatter_template = "Write a {word_count} word essay about {topic}."

prompt = PromptTemplate(
    input_variables=["word_count", "topic"], template=formatter_template
)

content_formatter = DollyContentFormatter()

llm = AzureMLOnlineEndpoint(
    endpoint_api_key=os.getenv("DOLLY_ENDPOINT_API_KEY"),
    endpoint_url=os.getenv("DOLLY_ENDPOINT_URL"),
    model_kwargs={"temperature": 0.8, "max_tokens": 300},
    content_formatter=content_formatter,
)

chain = LLMChain(llm=llm, prompt=prompt)
print(chain.invoke({"word_count": 100, "topic": "how to make friends"}))

API 參考文檔：LLMChain | DollyContentFormatter | PromptTemplate

序列化 LLM

您也可以儲存和載入 LLM 設定

from langchain_community.llms.loading import load_llm

save_llm = AzureMLOnlineEndpoint(
    deployment_name="databricks-dolly-v2-12b-4",
    model_kwargs={
        "temperature": 0.2,
        "max_tokens": 150,
        "top_p": 0.8,
        "frequency_penalty": 0.32,
        "presence_penalty": 72e-3,
    },
)
save_llm.save("azureml.json")
loaded_llm = load_llm("azureml.json")

print(loaded_llm)

API 參考文檔：load_llm

LLM 概念指南
LLM 操作指南

設定 (Set up)

內容格式器 (Content Formatter)

範例 (Examples)

範例：具有即時端點的 LlaMa 2 完成 (completions)

範例：具有隨用隨付部署的聊天完成 (Chat completions)（模型即服務）

範例：自定義內容格式器 (Custom content formatter)

範例：搭配 LLMChain 的 Dolly​

序列化 LLM​

相關資源​

此頁面是否對您有幫助？

範例：搭配 LLMChain 的 Dolly

序列化 LLM

相關資源