当前位置: 首页 > news >正文

VLLM 格式化LLM输出

文章目录

    • 前言
    • guided_json
    • guided_choice
    • guided_regex
    • guided_grammar
    • 总结

前言

vllm OpenAI Compatible Server 提供了格式化LLM输出的能力,默认的格式化解码后端应该是outlines

目前提供了四个参数来控制格式化输出,分别是:

guided_json: 按照给定的json schema输出
guided_choice: 从给定的选项里面选一个
guided_regex: 按照给定的正则表达式输出
guided_grammar: 按照给定的 扩展巴科斯范式(EBNF)格式 的上下文无关语法输出(我也不懂)

下面我们直接看看如何使用这四个参数,控制LLM的输出

import json
from openai import OpenAIdef chatgpt_base(system_prompt, user_prompt):api_key = "empty"base_url = "http://localhost:8000/v1"model = "Qwen1.5-14B-Chat-AWQ"client = OpenAI(api_key=api_key, base_url=base_url)completion = client.chat.completions.create(model=model,temperature=0,messages=[{"role": "system", "content": system_prompt},{"role": "user", "content": user_prompt},],)return completion.choices[0].message.content

guided_json

from pydantic import BaseModelclass Topic(BaseModel):问题: str答案: strdef chatgpt_guide_json(system_prompt, user_prompt):api_key = "empty"base_url = "http://localhost:8000/v1"model = "Qwen1.5-14B-Chat-AWQ"client = OpenAI(api_key=api_key, base_url=base_url)completion = client.chat.completions.create(model=model,temperature=0,messages=[{"role": "system", "content": system_prompt},{"role": "user", "content": user_prompt},],extra_body={"guided_json": Topic.model_json_schema()},)return completion.choices[0].message.contentsystem_prompt = "You are a helpful assistant."
user_prompt = """
请你生成一对和python相关的问题和答案
"""
response = chatgpt_base(system_prompt, user_prompt)
print("base response: ", response)
print("----" * 5)
guide_reponse = chatgpt_guide_json(system_prompt, user_prompt)
print("guide json reponse: ", guide_reponse)

输出:

base response:  问题:如何在Python中安装一个新的库?答案:在Python中,你可以使用pip工具来安装新的库。首先,你需要确保pip已安装。然后,打开命令行或终端,输入以下命令来安装所需的库:```pip install 库名```例如,如果你想安装requests库,你可以输入:```pip install requests```这将从Python Package Index (PyPI)下载并安装requests库及其依赖项。
--------------------
guide json reponse:  { "问题": "如何在Python中安装一个新的库?", "答案": "在Python中,你可以使用pip工具来安装新的库。例如,如果你想安装requests库,你可以在命令行中输入:\">> pip install requests\"。这将会从Python Package Index (PyPI) 下载并安装requests库。" }

可以看到,即使我们不显式地在prompt中告诉LLM要返回JSON格式,我们拿到的响应竟还是JSON,并且符合我们给的格式。

我们还能给每个字段添加解释,如:

class Topic(BaseModel):问题: str = Field(description="问题")答案: str = Field(description="答案")

guided_choice

def chatgpt_guide_choice(system_prompt, user_prompt):api_key = "empty"base_url = "http://localhost:8000/v1"model = "Qwen1.5-14B-Chat-AWQ"client = OpenAI(api_key=api_key, base_url=base_url)completion = client.chat.completions.create(model=model,temperature=0,messages=[{"role": "system", "content": system_prompt},{"role": "user", "content": user_prompt},],extra_body={"guided_choice": ["Positive", "Negative"]},)return completion.choices[0].message.contentsystem_prompt = "You are a helpful assistant."
user_prompt = """
Is the following review positive or negative?Review: 今天天气真不错,好想出去吃大餐
"""
response = chatgpt_base(system_prompt, user_prompt)
print("base response: ", response)
print("----" * 5)
guide_reponse = chatgpt_guide_choice(system_prompt, user_prompt)
print("guide choice reponse: ", guide_reponse)

输出:

base response:  这条评论并不是在评价某个产品或服务,而是在描述天气并表达了想出去吃大餐的愿望。不过,如果要从情感色彩来看,这条评论是积极的,因为它提到了好天气,并且表达了积极的愿望。
--------------------
guide choice reponse:  Positive

注意:guide_choice 无法输出选项中的多个答案,即无法处理多标签任务

guided_regex

def chatgpt_guide_regex(system_prompt, user_prompt):api_key = "empty"base_url = "http://localhost:8000/v1"model = "Qwen1.5-14B-Chat-AWQ"client = OpenAI(api_key=api_key, base_url=base_url)completion = client.chat.completions.create(model=model,temperature=0,messages=[{"role": "system", "content": system_prompt},{"role": "user", "content": user_prompt},],extra_body={"guided_regex": r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)"},)return completion.choices[0].message.contentsystem_prompt = "You are a helpful assistant."
user_prompt = """
What is the IP address of the Google DNS servers? 
"""
response = chatgpt_base(system_prompt, user_prompt)
print("base response: ", response)
print("----" * 5)
guide_reponse = chatgpt_guide_regex(system_prompt, user_prompt)
print("guide regex reponse: ", guide_reponse)

输出:

base response:  The IP addresses for the Google Public DNS servers are as follows:- 8.8.8.8 (primary server)
- 8.8.4.4 (secondary server)These can be used as your DNS servers to take advantage of Google's DNS service.
--------------------
guide regex reponse:  1.8.8.88

从这个输出结果来看,使用格式化输出似乎会导致LLM效果下降?

guided_grammar

arithmetic_grammar = r"""?start: expression?expression: term (("+" | "-") term)*?term: factor (("*" | "/") factor)*?factor: NUMBER| "-" factor| "(" expression ")"%import common.NUMBER
"""def chatgpt_guide_grammar(system_prompt, user_prompt):api_key = "empty"base_url = "http://localhost:8000/v1"model = "Qwen1.5-14B-Chat-AWQ"client = OpenAI(api_key=api_key, base_url=base_url)completion = client.chat.completions.create(model=model,temperature=0,messages=[{"role": "system", "content": system_prompt},{"role": "user", "content": user_prompt},],extra_body={"guided_grammar": arithmetic_grammar},)return completion.choices[0].message.contentsystem_prompt = "You are a helpful assistant."
user_prompt = """
Alice had 4 apples and Bob ate 2. Write an expression for Alice's apples:
"""
response = chatgpt_base(system_prompt, user_prompt)
print("base response: ", response)
print("----" * 5)
guide_reponse = chatgpt_guide_grammar(system_prompt, user_prompt)
print("guide grammar reponse: ", guide_reponse)

输出:

base response:  If Alice originally had 4 apples and Bob ate 2 of them, the expression for the number of apples Alice has left would be:\[ 4 - 2 \]So, Alice now has:\[ 4 - 2 = 2 \]Therefore, the expression for the number of apples Alice has left is \( 4 - 2 \).
--------------------
guide grammar reponse: (4-2)

这个咱也不懂,就不乱讲了,各位同学可以自行探索

感兴趣的同学可以看看:EBNF

总结

输出JSON还可以通过 response_format 控制,具体介绍可以查看vllm官方文档

这几个例子,也可以通过 outlines 仓库学习具体的用法


http://www.mrgr.cn/news/78317.html

相关文章:

  • c#基础练习71-75
  • TypeError: issubclass() arg 1 must be a class
  • Spring Bean初始化流程
  • Prometheus从二进制部署迁移Docker中更新到v3.0.0版本
  • ACP科普:易混概念“质量标准”与“DOD”
  • kali Linux中foremost安装
  • sed
  • 1、SpringBoo中Mybatis多数据源动态切换
  • Tomcat(36)Tomcat的静态资源缓存
  • docker-compose文件的简介及使用
  • C++虚函数面试题及参考答案
  • 【vue2】封装自定义的日历组件(一)之基础的组件结构
  • Educator头歌:离散数学 - 图论
  • 【机器学习】机器学习的基本分类-监督学习(Supervised Learning)
  • Swift——自动引用计数ARC
  • Javascript Insights: Visualizing Var, Let, And Const In 2024
  • Hbase2.2.7集群部署
  • 【不定长滑动窗口】【灵神题单】【刷题笔记】
  • 【拥抱AI】RAG如何通过分析反馈、识别问题来提高命中率
  • 探索.NET世界的无限可能——带你轻松了解.NET
  • Scala—Map用法详解
  • 图元交互设计
  • 【去毛刺】OpenCV图像处理基础:腐蚀与膨胀操作入门
  • 365天深度学习训练营-第P6周:VGG-16算法-Pytorch实现人脸识别
  • digit_eye开发记录(2): Python读取MNIST数据集
  • 大语言模型LLM的微调中 QA 转换的小工具 txt2excel.py