Questions about speech-truncation-detection-12M for audiobook/TTS QC
Hi @brthor — I'm work on an open-source audio QC tool called FinalPass (on my github at https://github.com/alexeymohr/FinalPass) and came across speech-truncation-detection-12M. Looks like it could really help with a problem I want to solve: detecting TTS utterances where the final word remains intelligible, but the final phoneme/decay has been audibly chopped off.
I have a real labeled test case from an AI-generated audiobook with four independently identified examples of this defect, plus roughly 110 clean utterance endings, so I'd love to evaluate your model against it.
Before doing that, a few quick questions:
- What kind of truncations was the model trained to detect? Especially whether it includes subtle terminal-phoneme truncation versus more obvious mid-word cuts? My problem audio is intelligible, but the very ends of words get chopped.
- Were the positive examples naturally occurring, synthetically truncated, or both?
- What license do you intend for the model code and weights? I couldn't find one declared in the repository. FinalPass is an open-source GitHub project, so I want to be careful and not integrate anything that isn't allowed.
Happy to share the results of my evaluation back with you. Your model is the first purpose-built system I've found that appears to target this exact class of problem.
Hello @alexeymohr
In my testing the detection is fairly sensitive. It won't detect if a word itself is incomplete (not a language model), but if the audio ends abruptly in a word or as a word is beginning it will mark that.
IIRC it was trained using forced alignment timings with something like 20-30ms of sensitivity.
apache 2.0 license has been added
I have made wide use of this model. Any AI agent should be able to make short work of an eval given the examples. I'd be happy to hear if you end up using it.
Thanks for taking a look.
EDIT:
My problem audio is intelligible, but the very ends of words get chopped.
This is the problem the model was built to detect, for the purpose of improving TTS training datasets.