soninho-voice — OpenVoice v2 speaker encoder (ONNX)

An ONNX export of the speaker (tone colour) encoder from OpenVoice v2 by MyShell.ai, exported from the official converter/checkpoint.pth and verified against the reference implementation.

Used by Soninho Mágico to let a family enrol a loved one's voice on device: the recording never leaves the phone, and only the resulting 256-float embedding is kept.

Attribution

The model weights are the work of MyShell.ai, released under the MIT licence and reproduced here in full in LICENSE. This repository contains only a format conversion — no retraining, no fine-tuning, no change to the weights' values.

• Upstream weights: myshell-ai/OpenVoiceV2
• Upstream code: myshell-ai/OpenVoice
• Paper: OpenVoice: Versatile Instant Voice Cloning

Files

file	size	description
tone_extract.onnx	7.5 MB	audio → 256-float speaker embedding

Interface

input   audio      float32  [1, N]      mono, 22050 Hz, N dynamic
output  embedding  float32  [1, 256, 1]

The STFT is baked into the graph — the caller passes raw samples and does no signal processing of its own. Audio must be at 22050 Hz; the encoder does not resample, and feeding it another rate returns a confident embedding for a pitch-shifted voice rather than an error.

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("tone_extract.onnx")
embedding = sess.run(None, {"audio": audio_22050[None, :].astype(np.float32)})[0]
# -> (1, 256, 1)

How it was exported

Two details that are easy to get wrong, recorded because they cost real time:

1. torch.onnx.export(..., dynamo=False). Torch 2.13's default dynamo exporter fails to decompose the reflect padding under a dynamic axis. The legacy TorchScript path handles it.
2. The STFT is a conv1d Fourier basis, not torch.stft. OpenVoice already ships this as spectrogram_torch_conv in openvoice/mel_processing.py, precisely because torch.stft exports badly.

Constants: n_fft 1024, hop 256, win 1024, magnitude sqrt(re² + im² + 1e-6), opset 17.

Verification

Cosine similarity against the reference ToneColorConverter.extract_se on the same audio:

torch wrapper vs official extract_se: cos=1.000000
ONNX        vs official extract_se: cos=1.000000

Anything below ~0.999 means the graph is wrong rather than merely imprecise.

Intended use and limits

Built for consented voice enrolment in a children's bedtime-story app — a grandparent recording their own voice so it can read stories aloud.

The MIT licence imposes no use restrictions. The following are our requirements, not the licence's, and are stated here because anyone reusing this should think about them:

• Get the speaker's consent, not the consent of whoever is holding the phone. Voice used for identification is biometric data; under Brazil's LGPD (art. 11) it is sensitive personal data, and children's data (art. 14) needs specific parental consent on top.
• Disclose synthetic speech. Audio produced this way should not be presented as a genuine recording of a real person.
• This model only produces an embedding. It cannot synthesise speech on its own, and the embedding cannot be inverted back into the original recording.