Llamafile
Llamafile 讓您可以使用單一檔案發布和執行 LLM。
Llamafile 透過將 llama.cpp 與 Cosmopolitan Libc 結合到一個框架中來達成此目的，該框架將 LLM 的所有複雜性簡化為可在大多數電腦本機執行的單一檔案執行檔（稱為「llamafile」），無需安裝。
設定
- 下載您想使用的模型的 llamafile。您可以在 HuggingFace 上找到許多 llamafile 格式的模型。在本指南中，我們將下載一個小的模型，
TinyLlama-1.1B-Chat-v1.0.Q5_K_M。注意：如果您沒有wget，您可以透過此連結下載模型。
wget https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile
- 使 llamafile 成為可執行檔。首先，如果您尚未執行此操作，請開啟終端機。如果您使用的是 MacOS、Linux 或 BSD，您需要使用
chmod授予您的電腦執行此新檔案的權限（請參閱下文）。如果您使用的是 Windows，請將檔案重新命名，在結尾新增「.exe」（模型檔案應命名為TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile.exe）。
chmod +x TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile # run if you're on MacOS, Linux, or BSD
- 在「伺服器模式」下執行 llamafile
./TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile --server --nobrowser
現在您可以呼叫 llamafile 的 REST API。依預設，llamafile 伺服器會在 https://127.0.0.1:8080 監聽。您可以在此處找到完整的伺服器文件。您可以透過 REST API 直接與 llamafile 互動，但在此我們將展示如何使用 LangChain 與其互動。
使用方式
from langchain_community.llms.llamafile import Llamafile
llm = Llamafile()
llm.invoke("Tell me a joke")
API 參考：Llamafile
'? \nI\'ve got a thing for pink, but you know that.\n"Can we not talk about work anymore?" - What did she say?\nI don\'t want to be a burden on you.\nIt\'s hard to keep a good thing going.\nYou can\'t tell me what I want, I have a life too!'
若要串流 token，請使用 .stream(...) 方法
query = "Tell me a joke"
for chunks in llm.stream(query):
print(chunks, end="")
print()
.
- She said, "I’m tired of my life. What should I do?"
- The man replied, "I hear you. But don’t worry. Life is just like a joke. It has its funny parts too."
- The woman looked at him, amazed and happy to hear his wise words. - "Thank you for your wisdom," she said, smiling. - He replied, "Any time. But it doesn't come easy. You have to laugh and keep moving forward in life."
- She nodded, thanking him again. - The man smiled wryly. "Life can be tough. Sometimes it seems like you’re never going to get out of your situation."
- He said, "I know that. But the key is not giving up. Life has many ups and downs, but in the end, it will turn out okay."
- The woman's eyes softened. "Thank you for your advice. It's so important to keep moving forward in life," she said. - He nodded once again. "You’re welcome. I hope your journey is filled with laughter and joy."
- They both smiled and left the bar, ready to embark on their respective adventures.
若要深入瞭解 LangChain Expressive Language 和 LLM 上可用的方法，請參閱LCEL 介面