Text-to-Speech
ONNX
F5-TTS
phoonnx

f5tts-multi-sema_afro-unicode

F5-TTS voice for phoonnx, converted to ONNX from travis-moore/sema-tts-afro.

Model details

Field Value
Repository Phoonnx/f5tts-multi-sema_afro-unicode
Voice id f5tts/sema-afro
Languages mul
Architecture F5-TTS
Text input The characters of the text, capitals and marks included (f5tts_pinyin)
Punctuation Trained with the clause punctuation , . ! ? : ;. phoonnx 1.96.1a1 and later keep it at synthesis.
Sample rate 24000 Hz
Speakers 1
Training code Upstream training code; see travis-moore/sema-tts-afro
Phonemizer scriptconv f5tts_pinyin (0.0.4a110 and later), a port of the F5-TTS function convert_char_to_pinyin. It reads a Chinese character as its pinyin syllable
Training data Not recorded upstream
Base model mrfakename/OpenF5-TTS
Training checkpoint None in this repository
Upstream travis-moore/sema-tts-afro
Trained by travis-moore (Sema) (upstream)
Licence Apache 2.0

QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The reference clip is LJ Speech LJ001-0009 (public domain). QA covers English only. The QA ran with the unicode text input. On its three sentences, that input and f5tts_pinyin give the same ids.

Text ids: phoonnx gives the model the characters of the text, marks included, as the F5-TTS text function does. The text keeps the Unicode form of each letter, so a precomposed Yoruba letter with a tone mark is one id. On FLORES-200 rows of the card languages, the ids equal the F5-TTS ids on 183 of 198 rows. On 6 more rows, the ids of each sentence are equal: phoonnx runs the model once for each sentence. On the other 9 rows, the only difference is a space or a no-break space at the start or the end of the row. F5-TTS inference also removes it.

Check fp32 int8
Result pass pass
WER 0.038 0.000
Characters per second 13.8 to 13.8 -
Length, int8 / fp32 - 1.00 to 1.00
Graph size 1455.8 MB 463.3 MB
Peak RSS 3200.7 MB 1204.2 MB
RTF (median of 3, 4 threads) 29.445 22.578

Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.

Each <graph>_int8.onnx file is the int8 copy of <graph>.onnx, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.

How to use

Install phoonnx:

pip install --pre phoonnx

Then run:

import wave
from phoonnx.config import SynthesisConfig
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["f5tts/sema-afro"].load()

with wave.open("output.wav", "wb") as wav_file:
    # The voice clones the speaker of a short clip. Give the clip and its transcription.
    syn_config = SynthesisConfig(speaker_reference="reference.wav",
                                 speaker_reference_text="What the speaker says in the clip.")
    voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file, syn_config=syn_config)

In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select this voice in mycroft.conf:

{
  "tts": {
    "module": "ovos-tts-plugin-phoonnx",
    "ovos-tts-plugin-phoonnx": {
      "voice": "f5tts/sema-afro"
    }
  }
}

Prepare the text

Run these lines on the text before it goes to voice.synthesize_wav:

# uv pip install --prerelease=allow text2spoken
from text2spoken import normalize

text = normalize(text, "en")  # the tag of the language of the text

text2spoken writes numbers, dates, times, money, units and abbreviations as the words that a speaker says. This voice does not name one language, so give text2spoken the tag of the language of the text.

Training data

The upstream card does not name the training data. See travis-moore/sema-tts-afro.

Related links

Ownership and licensing

travis-moore (Sema) trained this model and publishes it at travis-moore/sema-tts-afro under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit travis-moore (Sema) when you use this voice.

Downloads last month
284
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Phoonnx/f5tts-multi-sema_afro-unicode

Quantized
(1)
this model

Collection including Phoonnx/f5tts-multi-sema_afro-unicode