Instructions to use openbmb/VoxCPM2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- VoxCPM
How to use openbmb/VoxCPM2 with VoxCPM:
import soundfile as sf from voxcpm import VoxCPM model = VoxCPM.from_pretrained("openbmb/VoxCPM2") wav = model.generate( text="VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech.", prompt_wav_path=None, # optional: path to a prompt speech for voice cloning prompt_text=None, # optional: reference text cfg_value=2.0, # LM guidance on LocDiT, higher for better adherence to the prompt, but maybe worse inference_timesteps=10, # LocDiT inference timesteps, higher for better result, lower for fast speed normalize=True, # enable external TN tool denoise=True, # enable external Denoise tool retry_badcase=True, # enable retrying mode for some bad cases (unstoppable) retry_badcase_max_times=3, # maximum retrying times retry_badcase_ratio_threshold=6.0, # maximum length restriction for bad case detection (simple but effective), it could be adjusted for slow pace speech ) sf.write("output.wav", wav, 16000) print("saved: output.wav") - Notebooks
- Google Colab
- Kaggle
Speed issue in results.
Thanks a lot for releasing this, I was waiting for a new VOX for quite some time.
There is an issue however, I give it around 20-30 secs of reference audio with full transcription (ultimate mode) and no matter what I generate, speed is always 2 or 3 times faster than it should be with incredibly good quality. Is this is known issue? Kind regards.
I don't think anyone else has reported a similar problem before, because ultimate mode cloning usually maintains the same speed as the reference audio. Could it be due to the sampling rate of the saved audio?
Thanks a lot for releasing this, I was waiting for a new VOX for quite some time.
There is an issue however, I give it around 20-30 secs of reference audio with full transcription (ultimate mode) and no matter what I generate, speed is always 2 or 3 times faster than it should be with incredibly good quality. Is this is known issue? Kind regards.
Faced the same issue on a run. Changing the seed (to 0 in that case) helped with the pacing.
Faced the same issue on a run.
Worth splitting the diagnosis with one check: does the pitch shift along with the speed?
If the voice also sounds higher, it is a sample rate mismatch between what the model writes and what the player assumes, and 24k played back as 48k gives exactly a 2x. If the pitch is unchanged and only the pacing is wrong, the sample rate is fine and the duration conditioning on the reference is the place to look, which would also fit drakexp finding that a different seed changes it.