发布日期:2025年12月5日
VoxCPM1.5在音频质量和效率方面带来了提升:
| 特性 | VoxCPM | VoxCPM1.5 |
|---|---|---|
| 音频VAE采样率 | 16kHz | 44.1kHz |
| 语言模型令牌速率 | 12.5Hz | 6.25Hz |
| 补丁大小 | 2 | 4 |
| SFT支持 | ✅ | ✅ |
| LoRA支持 | ✅ | ✅ |
主要改进:
注意:输出质量取决于提示语音质量。VoxCPM-0.5B仍完全受支持,并保持向后兼容性。
VoxCPM是一款创新的无分词器文本转语音(TTS)系统,重新定义了语音合成的真实感。通过在连续空间中对语音进行建模,它克服了离散分词的局限性,实现了两项旗舰功能:上下文感知语音生成和高保真零样本语音克隆。
与将语音转换为离散令牌的主流方法不同,VoxCPM采用端到端扩散自回归架构,直接从文本生成连续语音表示。它基于MiniCPM-4骨干构建,通过分层语言建模和FSQ约束实现隐式语义-声学解耦,显著提升了表现力和生成稳定性。
pip install voxcpm默认情况下,首次运行脚本时会自动下载模型,您也可以提前手动下载模型。
下载 VoxCPM1.5
from huggingface_hub import snapshot_download
snapshot_download("openbmb/VoxCPM1.5")或下载 VoxCPM-0.5B
from huggingface_hub import snapshot_download
snapshot_download("openbmb/VoxCPM-0.5B")下载 ZipEnhancer 和 SenseVoice-Small。我们在网页演示中使用 ZipEnhancer 来增强语音提示,并使用 SenseVoice-Small 进行语音提示的语音识别(ASR)。
from modelscope import snapshot_download
snapshot_download('iic/speech_zipenhancer_ans_multiloss_16k_base')
snapshot_download('iic/SenseVoiceSmall')import soundfile as sf
import numpy as np
from voxcpm import VoxCPM
model = VoxCPM.from_pretrained("openbmb/VoxCPM1.5")
# Non-streaming
wav = model.generate(
text="VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech.",
prompt_wav_path=None, # optional: path to a prompt speech for voice cloning
prompt_text=None, # optional: reference text
cfg_value=2.0, # LM guidance on LocDiT, higher for better adherence to the prompt, but maybe worse
inference_timesteps=10, # LocDiT inference timesteps, higher for better result, lower for fast speed
normalize=False, # enable external TN tool, but will disable native raw text support
denoise=False, # enable external Denoise tool, but it may cause some distortion and restrict the sampling rate to 16kHz
retry_badcase=True, # enable retrying mode for some bad cases (unstoppable)
retry_badcase_max_times=3, # maximum retrying times
retry_badcase_ratio_threshold=6.0, # maximum length restriction for bad case detection (simple but effective), it could be adjusted for slow pace speech
)
sf.write("output.wav", wav, model.tts_model.sample_rate)
print("saved: output.wav")
# Streaming
chunks = []
for chunk in model.generate_streaming(
text = "Streaming text to speech is easy with VoxCPM!",
# supports same args as above
):
chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("output_streaming.wav", wav, model.tts_model.sample_rate)
print("saved: output_streaming.wav")安装完成后,入口点为 voxcpm(或使用 python -m voxcpm.cli)。
# 1) Direct synthesis (single text)
voxcpm --text "VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech." --output out.wav
# 2) Voice cloning (reference audio + transcript)
voxcpm --text "VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech." \
--prompt-audio path/to/voice.wav \
--prompt-text "reference transcript" \
--output out.wav \
# --denoise
# (Optinal) Voice cloning (reference audio + transcript file)
voxcpm --text "VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech." \
--prompt-audio path/to/voice.wav \
--prompt-file "/path/to/text-file" \
--output out.wav \
# --denoise
# 3) Batch processing (one text per line)
voxcpm --input examples/input.txt --output-dir outs
# (optional) Batch + cloning
voxcpm --input examples/input.txt --output-dir outs \
--prompt-audio path/to/voice.wav \
--prompt-text "reference transcript" \
# --denoise
# 4) Inference parameters (quality/speed)
voxcpm --text "..." --output out.wav \
--cfg-value 2.0 --inference-timesteps 10 --normalize
# 5) Model loading
# Prefer local path
voxcpm --text "..." --output out.wav --model-path /path/to/VoxCPM_model_dir
# Or from Hugging Face (auto download/cache)
voxcpm --text "..." --output out.wav \
--hf-model-id openbmb/VoxCPM1.5 --cache-dir ~/.cache/huggingface --local-files-only
# 6) Denoiser control
voxcpm --text "..." --output out.wav \
--no-denoiser --zipenhancer-path iic/speech_zipenhancer_ans_multiloss_16k_base
# 7) Help
voxcpm --help
python -m voxcpm.cli --help您可以通过运行python app.py启动UI界面,该界面支持您进行声音克隆和声音创建操作。
VoxCPM1.5支持全量微调(SFT)和LoRA微调,您可以使用自己的数据训练个性化声音模型。详细操作说明请参见微调指南。
快速开始:
# Full fine-tuning
python scripts/train_voxcpm_finetune.py \
--config_path conf/voxcpm_v1.5/voxcpm_finetune_all.yaml
# LoRA fine-tuning
python scripts/train_voxcpm_finetune.py \
--config_path conf/voxcpm_v1.5/voxcpm_finetune_lora.yaml欢迎来到 VoxCPM 厨房!按照这份食谱,你将能“烹饪”出完美的生成语音。让我们开始吧。
首先,选择你喜欢的文本输入方式:
这是赋予音频独特音效的“秘密配方”。
您已准备好“上菜”!但对于想要调整“风味”的大厨们,这里有两种关键“香料”。
祝您创作愉快!🎉 从默认设置开始,然后根据您的项目进行调整。厨房由您掌控!
VoxCPM模型权重和代码基于Apache-2.0许可证开源。