DeepSeek-V2.5
1. 简介
DeepSeek-V2.5 是 DeepSeek-V2-Chat 与 DeepSeek-Coder-V2-Instruct 的融合升级版本。新模型整合了前两个版本的通用能力与编程能力。 如需了解模型详情，请访问 DeepSeek-V2 页面 获取更多信息。
DeepSeek-V2.5 在人类偏好对齐方面表现更优，并在写作、指令遵循等多个维度进行了优化：
| 指标 | DeepSeek-V2-0628 | DeepSeek-Coder-V2-0724 | DeepSeek-V2.5 |
|---|---|---|---|
| AlpacaEval 2.0 | 46.6 | 44.5 | 50.5 |
| ArenaHard | 68.3 | 66.3 | 76.2 |
| AlignBench | 7.88 | 7.91 | 8.04 |
| MT-Bench | 8.85 | 8.91 | 9.02 |
| HumanEval python | 84.5 | 87.2 | 89 |
| HumanEval 多语言 | 73.8 | 74.8 | 73.8 |
| LiveCodeBench(01-09) | 36.6 | 39.7 | 41.8 |
| Aider | 69.9 | 72.9 | 72.2 |
| SWE-verified | 不适用 | 19 | 16.8 |
| DS-FIM-Eval | 不适用 | 73.2 | 78.3 |
| DS-Arena-Code | 不适用 | 49.5 | 63.1 |
2. 本地运行指南
使用 BF16 格式运行 DeepSeek-V2.5 推理需配备 80GB*8 张显卡。
通过 Huggingface Transformers 推理
您可以直接使用 Huggingface Transformers 进行模型推理。
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
model_name = "deepseek-ai/DeepSeek-V2.5"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
# `max_memory` should be set based on your devices
max_memory = {i: "75GB" for i in range(8)}
# `device_map` cannot be set to `auto`
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="sequential", torch_dtype=torch.bfloat16, max_memory=max_memory, attn_implementation="eager")
model.generation_config = GenerationConfig.from_pretrained(model_name)
model.generation_config.pad_token_id = model.generation_config.eos_token_id
messages = [
{"role": "user", "content": "Write a piece of quicksort code in C++"}
]
input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(input_tensor.to(model.device), max_new_tokens=100)
result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
print(result)完整的聊天模板可在 Hugging Face 模型仓库中的 tokenizer_config.json 文件内找到。
注意：相较于之前的 DeepSeek-V2-Chat 版本，聊天模板已更新。
聊天模板示例如下：
<｜begin▁of▁sentence｜><｜User｜>{user_message_1}<｜Assistant｜>{assistant_message_1}<｜end▁of▁sentence｜><｜User｜>{user_message_2}<｜Assistant｜>您还可以添加一个可选的系统消息：
<｜begin▁of▁sentence｜>{system_message}<｜User｜>{user_message_1}<｜Assistant｜>{assistant_message_1}<｜end▁of▁sentence｜><｜User｜>{user_message_2}<｜Assistant｜>使用 vLLM 进行推理（推荐）
要使用 vLLM 进行模型推理，请将此 Pull Request 合并到您的 vLLM 代码库中：https://github.com/vllm-project/vllm/pull/4650。
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
max_model_len, tp_size = 8192, 8
model_name = "deepseek-ai/DeepSeek-V2.5"
tokenizer = AutoTokenizer.from_pretrained(model_name)
llm = LLM(model=model_name, tensor_parallel_size=tp_size, max_model_len=max_model_len, trust_remote_code=True, enforce_eager=True)
sampling_params = SamplingParams(temperature=0.3, max_tokens=256, stop_token_ids=[tokenizer.eos_token_id])
messages_list = [
[{"role": "user", "content": "Who are you?"}],
[{"role": "user", "content": "Translate the following content into Chinese directly: DeepSeek-V2 adopts innovative architectures to guarantee economical training and efficient inference."}],
[{"role": "user", "content": "Write a piece of quicksort code in C++."}],
]
prompt_token_ids = [tokenizer.apply_chat_template(messages, add_generation_prompt=True) for messages in messages_list]
outputs = llm.generate(prompt_token_ids=prompt_token_ids, sampling_params=sampling_params)
generated_text = [output.outputs[0].text for output in outputs]
print(generated_text)函数调用
函数调用功能使模型能够调用外部工具以增强其能力。
示例如下：
# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
tool_system_prompt = """You are a helpful Assistant.
## Tools
### Function
You have the following functions available:
- `get_current_weather`:
```json
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit"
]
}
},
"required": [
"location"
]
}
}
```"""
tool_call_messages = [{"role": "system", "content": tool_system_prompt}, {"role": "user", "content": "What's the weather like in Tokyo and Paris?"}]
tool_call_inputs = tokenizer.apply_chat_template(tool_call_messages, add_generation_prompt=True, return_tensors="pt")
tool_call_outputs = model.generate(tool_call_inputs.to(model.device))
# Generated text: '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>get_current_weather\n```json\n{"location": "Tokyo"}\n```<｜tool▁call▁end｜>\n<｜tool▁call▁begin｜>function<｜tool▁sep｜>get_current_weather\n```json\n{"location": "Paris"}\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜><｜end▁of▁sentence｜>'
# Mock response of calling `get_current_weather`
tool_messages = [{"role": "tool", "content": '{"location": "Tokyo", "temperature": "10", "unit": null}'}, {"role": "tool", "content": '{"location": "Paris", "temperature": "22", "unit": null}'}]
tool_inputs = tokenizer.apply_chat_template(tool_messages, add_generation_prompt=False, return_tensors="pt")[:, 1:]
tool_inputs = torch.cat([tool_call_outputs, tool_inputs.to(model.device)], dim=1)
tool_outputs = model.generate(tool_inputs)
# Generated text: The current weather in Tokyo is 10 degrees, and in Paris, it is 22 degrees.<｜end▁of▁sentence｜>JSON 输出
您可以使用 JSON 输出模式来确保模型生成有效的 JSON 对象。要启用此模式，需要在系统提示中添加一条特殊指令。
# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
user_system_prompt = 'The user will provide some exam text. Please parse the "question" and "answer" and output them in JSON format.'
json_system_prompt = f"""{user_system_prompt}
## Response Format
Reply with JSON object ONLY."""
json_messages = [{"role": "system", "content": json_system_prompt}, {"role": "user", "content": "Which is the highest mountain in the world? Mount Everest."}]
json_inputs = tokenizer.apply_chat_template(json_messages, add_generation_prompt=True, return_tensors="pt")
json_outpus = model.generate(json_inputs.to(model.device))
# Generated text: '```json\n{\n "question": "Which is the highest mountain in the world?",\n "answer": "Mount Everest."\n}\n```<｜end▁of▁sentence｜>'FIM 补全
在 FIM（填充中间部分）补全任务中，您可以提供一个前缀和一个可选的后缀，模型将自动补全中间的内容。
# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
prefix = """def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[0]
left = []
right = []
"""
suffix = """
if arr[i] < pivot:
left.append(arr[i])
else:
right.append(arr[i])
return quick_sort(left) + [pivot] + quick_sort(right)"""
fim_prompt = f"<｜fim▁begin｜>{prefix}<｜fim▁hole｜>{suffix}<｜fim▁end｜>"
fim_inputs = tokenizer(fim_prompt, add_special_tokens=True, return_tensors="pt").input_ids
fim_outputs = model.generate(fim_inputs.to(model.device))
# Generated text: " for i in range(1, len(arr)):<｜end▁of▁sentence｜>"3. 许可声明
本代码仓库采用 MIT 许可证授权。使用 DeepSeek-V2 Base/Chat 模型需遵守模型许可协议。DeepSeek-V2 系列（包括 Base 和 Chat 版本）支持商业用途。
4. 引用说明
@misc{deepseekv2,
title={DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model},
author={DeepSeek-AI},
year={2024},
eprint={2405.04434},
archivePrefix={arXiv},
primaryClass={cs.CL}
}5. 联系我们
如有任何疑问，请提交问题或通过邮件 service@deepseek.com 与我们联系。