VideoJev-35B-A3B

基于 Qwen/Qwen3.6-35B-A3B 的视频系统一（System One）模型，通过直接候选项评分实现时序定位与多选视频问答。

基准测试结果

VideoJev-35B-A3B 时序定位质量与延迟

VideoJev-35B-A3B 视频问答质量与延迟

快速开始

已在 Python 3.12、PyTorch 2.8 / CUDA 12.8 及 H200 上测试通过。请先安装对应的 PyTorch 版本。

pip install transformers==5.14.1 accelerate safetensors numpy pillow av==18.1.0 \
  flash-linear-attention==0.4.2 fla-core==0.4.2
pip install flash-attn causal-conv1d --no-build-isolation

对吃意大利面（10秒）执行定位与问答，帧率为 2 FPS。

import json
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoProcessor

video = hf_hub_download(
    repo_id="ZhuYuhan/VideoJev-Demo", filename="eating_spaghetti.mp4",
    repo_type="dataset", revision="7e2b9685bca592c3b929268f6fcf82c4ad74b267",
)
repo = "ZhuYuhan/VideoJev-35B-A3B"
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16,
    device_map={"": "cuda:0"}, attn_implementation="flash_attention_2",
)
model.prepare_for_inference()

questions = [
    {"id": "grounding-1", "task": "grounding",
     "query": "The woman in green eats pasta."},
    {"id": "qa-1", "task": "choice",
     "query": "What is the woman in green eating?",
     "options": ["A. Pasta.", "B. Rice.", "C. Soup."]},
]
results = model.predict_video(
    processor, video, questions,
    fps=2.0, max_frames=2048, context=8192,
)
print(json.dumps(results, indent=2))

输出

上述视频和代码的实际输出：

[
  {
    "id": "grounding-1",
    "task": "grounding",
    "probabilities": [
      0.0029175332747399807,
      0.008276241831481457,
      0.012840493582189083,
      0.8461092710494995,
      0.9874667525291443,
      0.9924765229225159,
      0.9928001165390015,
      0.9841585159301758,
      0.15136416256427765,
      0.029093489050865173
    ],
    "interval": [
      3.0,
      8.0
    ]
  },
  {
    "id": "qa-1",
    "task": "choice",
    "probabilities": [
      0.9999183416366577,
      5.838637662236579e-05,
      2.327314177819062e-05
    ],
    "answer_index": 0,
    "answer": "A. Pasta."
  }
]

interval 的单位为秒；answer_index 从零开始计数。probabilities 包含用于定位的独立 sigmoid 分数以及 QA 选项上的 softmax 分数。

定位默认 threshold=0.25；向 predict_video 或 predict_features 传入 threshold=0.4 可在单次调用中覆盖该值。

有关特定任务的配置和输出详情，请参阅使用指南。

Citation

@article{zhu2026videojev,
  title={VideoJev: System One Model for Video Understanding and Interaction},
  author={Zhu, Yuhan and Zeng, Xiangyu and Si, Qingyi and Yang, Chenxu and Li, Yifei and Qin, Luozheng and Zhou, Junhao and Yao, Dingyu and Zhang, Jun and Qin, Chuanyu and others},
  journal={arXiv preprint arXiv:2610.18615},
  year={2026}
}