Qwen3.5-35B-A3B

简介

Qwen3.5 具备以下增强功能：

统一的视觉-语言基础：通过多模态令牌的早期融合训练，实现了与 Qwen3 跨代性能的持平，并在推理、编程、智能体和视觉理解等基准测试中优于 Qwen3-VL 模型。
高效的混合架构：门控差分网络（Gated Delta Networks）结合稀疏专家混合（Sparse Mixture-of-Experts）架构，实现了高吞吐量的推理，同时将延迟和成本开销降至最低。
可扩展的强化学习泛化能力：在百万智能体环境中进行强化学习，逐步增加任务复杂度，从而实现对现实世界环境的稳健适应。
全球语言覆盖：扩展支持至201种语言和方言，实现包容性的全球部署，具备对不同文化和地区的细致理解。
下一代训练基础设施：与纯文本训练相比，多模态训练效率接近100%，并采用异步强化学习框架，支持大规模智能体架构和环境编排。

环境准备

安装

NPU运行时环境所需的依赖已集成到Docker镜像中，并上传至quay.io平台，用户可直接拉取该镜像。

#Atlas 800 A3
docker pull quay.io/ascend/sglang:v0.5.9-cann8.5.0-a3
#Atlas 800 A2
docker pull quay.io/ascend/sglang:v0.5.9-cann8.5.0-910b

#start container
docker run -itd --shm-size=16g --privileged=true --name ${NAME} \
--privileged=true --net=host \
-v /var/queue_schedule:/var/queue_schedule \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /usr/local/sbin:/usr/local/sbin \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware \
--device=/dev/davinci0:/dev/davinci0  \
--device=/dev/davinci1:/dev/davinci1  \
--device=/dev/davinci2:/dev/davinci2  \
--device=/dev/davinci3:/dev/davinci3  \
--device=/dev/davinci4:/dev/davinci4  \
--device=/dev/davinci5:/dev/davinci5  \
--device=/dev/davinci6:/dev/davinci6  \
--device=/dev/davinci7:/dev/davinci7  \
--device=/dev/davinci8:/dev/davinci8  \
--device=/dev/davinci9:/dev/davinci9  \
--device=/dev/davinci10:/dev/davinci10  \
--device=/dev/davinci11:/dev/davinci11  \
--device=/dev/davinci12:/dev/davinci12  \
--device=/dev/davinci13:/dev/davinci13  \
--device=/dev/davinci14:/dev/davinci14  \
--device=/dev/davinci15:/dev/davinci15  \
--device=/dev/davinci_manager:/dev/davinci_manager \
--device=/dev/hisi_hdc:/dev/hisi_hdc \
--entrypoint=bash \
quay.io/ascend/sglang:${tag}

权重下载

Model Name	address
Qwen3.5-35B-A3B	modelers
Qwen3.5-35B-A3B	modelscope
Qwen3.5-35B-A3B	gitcode

部署

单节点部署

执行以下脚本进行在线推理.

# high performance cpu
echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
sysctl -w vm.swappiness=0
sysctl -w kernel.numa_balancing=0
sysctl -w kernel.sched_migration_cost_ns=50000
# bind cpu
export SGLANG_SET_CPU_AFFINITY=1

unset https_proxy
unset http_proxy
unset HTTPS_PROXY
unset HTTP_PROXY
unset ASCEND_LAUNCH_BLOCKING
# cann
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh

export STREAMS_PER_DEVICE=32
export HCCL_BUFFSIZE=1000
export HCCL_OP_EXPANSION_MODE=AIV
export HCCL_SOCKET_IFNAME=lo
export GLOO_SOCKET_IFNAME=lo

python3 -m sglang.launch_server \
        --model-path $MODEL_PATH \
        --attention-backend ascend \
        --device npu \
        --tp-size 2 --nnodes 1 --node-rank 0 \
        --chunked-prefill-size 4096 --max-prefill-tokens 280000 \
        --disable-radix-cache \
        --trust-remote-code \
        --host 127.0.0.1 \
        --mem-fraction-static 0.7 \
        --port 8000 \
        --cuda-graph-bs 16 \
        --enable-multimodal \
        --mm-attention-backend ascend_attn \
        --dtype bfloat16

发送请求测试

curl --location http://127.0.0.1:8000/v1/chat/completions --header 'Content-Type: application/json' --data '{
  "model": "qwen3.5",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image_url",
          "image_url": {"url": "/image_path/qwen.png"} 
        },
        {"type": "text", "text": "What is the text in the illustrate?"}
      ]
    }
  ]
}'

结果返回如下

{"id":"cdcd6d14645846e69cc486554f198154","object":"chat.completion","created":1772098465,"model":"qwen3.5","choices":[{"index":0,"message":{"role":"assistant","content":"The user is asking about the text present in the image. I will analyze the image to identify the text.\n</think>\n\nThe text in the image is \"TONGyi Qwen\".","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":"stop","matched_stop":248044}],"usage":{"prompt_tokens":98,"total_tokens":138,"completion_tokens":40,"prompt_tokens_details":null,"reasoning_tokens":0},"metadata":{"weight_version":"default"}}

声明

1）当前仅为尝鲜体验，性能优化中。

2）本代码仓提到的数据集和模型仅作为示例，这些数据集和模型仅供您用于非商业目的，如您使用这些数据集和模型来完成示例，请您特别注意应遵守对应数据集和模型的License，如您因使用数据集或模型而产生侵权纠纷，华为不承担任何责任。

3）如您在使用本代码仓的过程中，发现任何问题（包括但不限于功能问题、合规问题），请在本代码仓提交issue，我们将及时审视并解答。