Instructions to use Phoonnx/f5tts-multi-sema_afro-unicode with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- F5-TTS
How to use Phoonnx/f5tts-multi-sema_afro-unicode with F5-TTS:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
f5tts-multi-sema_afro-unicode
F5-TTS voice for phoonnx, converted to ONNX from travis-moore/sema-tts-afro.
Model details
| Field | Value |
|---|---|
| Repository | Phoonnx/f5tts-multi-sema_afro-unicode |
| Voice id | f5tts/sema-afro |
| Languages | mul |
| Architecture | F5-TTS |
| Text input | The characters of the text, capitals and marks included (f5tts_pinyin) |
| Punctuation | Trained with the clause punctuation , . ! ? : ;. phoonnx 1.96.1a1 and later keep it at synthesis. |
| Sample rate | 24000 Hz |
| Speakers | 1 |
| Training code | Upstream training code; see travis-moore/sema-tts-afro |
| Phonemizer | scriptconv f5tts_pinyin (0.0.4a110 and later), a port of the F5-TTS function convert_char_to_pinyin. It reads a Chinese character as its pinyin syllable |
| Training data | Not recorded upstream |
| Base model | mrfakename/OpenF5-TTS |
| Training checkpoint | None in this repository |
| Upstream | travis-moore/sema-tts-afro |
| Trained by | travis-moore (Sema) (upstream) |
| Licence | Apache 2.0 |
QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The reference clip is LJ Speech LJ001-0009 (public domain). QA covers English only. The QA ran with the unicode text input. On its three sentences, that input and f5tts_pinyin give the same ids.
Text ids: phoonnx gives the model the characters of the text, marks included, as the F5-TTS text function does. The text keeps the Unicode form of each letter, so a precomposed Yoruba letter with a tone mark is one id. On FLORES-200 rows of the card languages, the ids equal the F5-TTS ids on 183 of 198 rows. On 6 more rows, the ids of each sentence are equal: phoonnx runs the model once for each sentence. On the other 9 rows, the only difference is a space or a no-break space at the start or the end of the row. F5-TTS inference also removes it.
| Check | fp32 | int8 |
|---|---|---|
| Result | pass | pass |
| WER | 0.038 | 0.000 |
| Characters per second | 13.8 to 13.8 | - |
| Length, int8 / fp32 | - | 1.00 to 1.00 |
| Graph size | 1455.8 MB | 463.3 MB |
| Peak RSS | 3200.7 MB | 1204.2 MB |
| RTF (median of 3, 4 threads) | 29.445 | 22.578 |
Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.
Each <graph>_int8.onnx file is the int8 copy of <graph>.onnx, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.
How to use
Install phoonnx:
pip install --pre phoonnx
Then run:
import wave
from phoonnx.config import SynthesisConfig
from phoonnx.model_manager import TTSModelManager
manager = TTSModelManager()
manager.load()
manager.merge_default_voices()
voice = manager.voices["f5tts/sema-afro"].load()
with wave.open("output.wav", "wb") as wav_file:
# The voice clones the speaker of a short clip. Give the clip and its transcription.
syn_config = SynthesisConfig(speaker_reference="reference.wav",
speaker_reference_text="What the speaker says in the clip.")
voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file, syn_config=syn_config)
In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select this voice in
mycroft.conf:
{
"tts": {
"module": "ovos-tts-plugin-phoonnx",
"ovos-tts-plugin-phoonnx": {
"voice": "f5tts/sema-afro"
}
}
}
Prepare the text
Run these lines on the text before it goes to voice.synthesize_wav:
# uv pip install --prerelease=allow text2spoken
from text2spoken import normalize
text = normalize(text, "en") # the tag of the language of the text
text2spoken writes numbers, dates, times, money, units and abbreviations as the words that a speaker says. This voice does not name one language, so give text2spoken the tag of the language of the text.
Training data
The upstream card does not name the training data. See travis-moore/sema-tts-afro.
Related links
- Upstream model: travis-moore/sema-tts-afro
- phoonnx: https://github.com/TigreGotico/phoonnx
- TigreGotico: https://tigregotico.pt
Ownership and licensing
travis-moore (Sema) trained this model and publishes it at travis-moore/sema-tts-afro under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit travis-moore (Sema) when you use this voice.
- Downloads last month
- 284