ampixa
← all posts
// post · khatraTTS

khatraTTS: a multilingual teacher for our small voices

2026-11-19 · Ampixa Labs · 7 min read

TL;DR. khatraTTS is our internal teacher text-to-speech model, FlowLM-based: a flow-matching language model over continuous audio frames with a continuous codec, about 324 million parameters and 17 languages, with later runs adding routes for languages of Nepal. We are building it to teach sanoTTS voices that are about a thousand times smaller. On the 15 newer languages its mean character error rate (CER) is .078, and 11 of 15 tested languages pass a two-sentence public check. These scores come from speech recognisers, and a listening test is next. This research model stays internal, and the teacher we are building on open data will be released openly.

khatraTTS, a FlowLM-based teacher text-to-speech model: about 324M parameters, 11 of 15 tested languages pass both test sentences, mean CER .078 on the 15 newer languages

One FlowLM-based network, 17 languages: 11 of 15 tested languages pass an automatic two-sentence check.

What does it sound like?

Every clip here is khatraTTS output from the 17-language model, passed through voice conversion (RVC) to a voice from our public Nepali RVC voice pack, so the clips do not carry the voices of the reference speakers used as prompts. Voice conversion changes the timbre, not the words or the timing. The CERs in the tables were measured before conversion.

The two sentences are items 1894 and 1674 of the FLEURS test set (Conneau et al., 2022), which carries the same sentence in each of these languages. The English translation of each sentence is the FLEURS English text. For each sentence we generated three samples, transcribed each with the MMS-1b-all recogniser (Pratap et al., 2023), and kept the sample with the lowest CER. A sentence whose best CER was above .15 was dropped.

Sentence A. In English: "A curry is a dish based on herbs and spices, together with either meat or vegetables."

LanguageClipTextCER before conversion
EnglishA curry is a dish based on herbs and spices, together with either meat or vegetables..000
ArmenianԿարրին մի ճաշատեսակ է, որի հիմնական բաղադրիչներն են խոտաբույսերն ու համեմունքները՝ պատրաստված կա՛մ մսով, կա՛մ բանջարեղենով:.135
GreekΤο κάρυ αποτελείται από μυρωδικά και καρυκεύματα, και συνοδεύεται με τροφές ζωικής ή φυτικής προέλευσης..078
MacedonianКарито е јадење со билки и зачини, што содржи месо или зеленчуци..048
MalayKari adalah hidangan berasaskan herba dan rempah yang dimasak dengan daging atau sayuran..045
FilipinoAng isang curry ay isang ulam na batay sa halaman at pampalasa, kasama ng alinman sa karne o gulay..031

Sentence A in six languages. Each player is khatraTTS output after voice conversion.

Sentence B. In English: "In some areas boiling water for a minute is enough, in others several minutes are needed."

LanguageClipTextCER before conversion
Nepaliकेही क्षेत्रहरूमा एक मिनेट पानी उमाल्न पर्याप्त छ, अरूमा धेरै मिनेट आवश्यक छ।.000
EnglishIn some areas boiling water for a minute is enough, in others several minutes are needed..000
ZuluKwezinye izindawo ukubilisa amanzi umzuzu kwanele, kwezinye kudingeka imizuzu eminingana..115
ArmenianՈրոշ տարածքներում ջուրը մի րոպե եռացնելը բավարար է լինում, ուրիշ տեղերում այդ անելու համար մի քանի րոպե է անհրաժեշտ:.114
MacedonianВо некои области, доволно е водата да врие една минута, додека во други, потребни се неколку минути..000
MalayDi beberapa kawasan, mendidih air selama seminit adalah memadai, di kawasan lain beberapa minit adalah diperlukan..045
FilipinoSa ilang mga lugar, ang pagpapakulo ng tubig sa loob ng isang minuto ay sapat na, sa iba, higit pa sa isang minuto ang kinakailangan..023

Sentence B in seven languages, Nepali and Zulu among them.

Team members listened to some of these clips before voice conversion. No listening panel has rated them. Credits for the voices are under Get it.

Why do tiny voices need a big teacher?

Because a very small model cannot learn to speak from raw recordings, and a big one can read to it.

Our smallest shipped voice, heart-nano, has 294,279 parameters and fits in 337 KB. Its larger sibling, heart, has 2.27M. Their word error rates are 0.083 and 0.080 (sanoTTS README). A model that small memorises its training text and fails on new sentences. What works is distillation: a larger model reads as much text as we want in one steady voice, and the small model learns to copy it. Our public distillation recipe does this with an open single-language teacher.

Parameter counts on a log scale: khatraTTS FlowLM 324,236,353, khatraTTS codec 20,054,337, sanoTTS heart 2.27M, sanoTTS heart-nano 294,279

The teacher is about a thousand times larger than the smallest voice it is meant to teach.

That recipe needs a teacher voice for each language. khatraTTS is our way to make one where no open teacher voice exists.

How is it built?

khatraTTS is a FlowLM-based teacher: a flow-matching language model over continuous audio frames, with a continuous codec.

Part What it does Size
Codec Compresses 24 kHz audio to 12.5 frames per second. Each frame is 32 continuous numbers, not discrete tokens. about 20 million parameters
FlowLM A 24-layer transformer, width 1024, reads the text and a few seconds of a reference voice. A small flow-matching head then produces the next audio frame. about 324 million parameters

Almost all of the model is the FlowLM.

Languages share 4,000-piece tokenizers and are told apart by a tag row in the text table. The latest run added six languages as six new rows, with the tokenizer unchanged.

The plan has three steps. The teacher reads a large, clean text set in one consistent voice per language. A sanoTTS student learns durations, acoustics and a decoder from that output. The student is exported to int8 for the browser and for microcontrollers.

We have not yet distilled a sanoTTS student from khatraTTS. That is the next stage of the work.

Which languages does it cover?

Seventeen in the model behind the clips here, and up to 54 routes in later runs. A route is one language tag the model can be asked to speak.

Model Coverage
17-language model English, Nepali, Afrikaans, Amharic, Azerbaijani, Bulgarian, Greek, Filipino, Armenian, Lithuanian, Macedonian, Mongolian, Malay, Romanian, Slovak, Albanian, Zulu
48-route run the 17 plus 31 routes for languages of Nepal, Newar among them
54-route run the 48 plus Limbu, Maithili, Sampang, Sunuwar, Eastern Tamang, Gurung

Three runs, from 17 languages to 54 routes.

The 54-route run is not yet better than the 48-route one on the older languages, so it is not the current model.

How intelligible is it on our own tests?

On held-out text the mean CER over the 15 newer languages is .078 with Scribe v2. CER is the share of characters a speech recogniser gets wrong when it transcribes the generated audio. Lower is better.

17-language model, clean reference voice Scribe v2 CER MMS-1b-all CER
Stock decoder .078 .140
Our decoder, step 10,000 .079 .129

Mean over the 15 newer languages, three texts per language and one sample per text. Our own decoder scores the same within noise.

  • Scribe CER is at or below .09 in 12 of the 15 languages. The exceptions are Amharic .23, Zulu .21 and Mongolian .17.
  • English scores .03 on Scribe.
  • Nepali scores 0 / .024 / 0 / .024 on four short sentences with our own Nepali recogniser.

For five of the languages the 54-route run adds, we compared generated speech with a human recording of the same sentence: MMS CER on 30 held-out texts and 2 seeds.

Language Human recording khatraTTS
Limbu .0458 .0483
Maithili .0256 .0360
Sampang .0309 .0327
Sunuwar .0218 .0279
Eastern Tamang .0142 .0207

Generated speech is recognised almost as well as the human recording in five languages of Nepal.

The held-out texts are close to the material the model was trained on, so read these numbers as optimistic.

How many languages pass a public-sentence check?

Eleven of the 15 we could test.

We ran the two-sentence check from the top of this post on 15 of the 17 languages; FLEURS has no Albanian, and its Afrikaans test split does not contain these items. 25 of the 30 sentences passed.

Dot chart of CER for 15 languages and two sentences each: three samples per sentence, the best in crimson, pass band up to .15. 25 of 30 sentences pass

25 of 30 sentences have a best sample inside the pass band.

11 languages passed on both sentences. If we take the middle sample of the three and not the best, 8 languages pass on both.

Coverage grid for the 17 languages: 11 pass both sentences, 3 pass one, 1 passes neither, 2 were not tested

Of 17 languages, 11 pass both sentences, 4 fall short and 2 could not be tested.

The other languages of Nepal are not in this check: for 30 of the 31 routes that the 48-route run adds, we found no public everyday test sentences.

What khatraTTS is not

  • Not released. The weights are internal research only.
  • Not rated by a listening panel yet. Every score here is from a speech recogniser.
  • Not yet distilled into a student.

Where this sits

What is next?

  • A listening test with native speakers.
  • A multi-sample benchmark: more texts, several seeds, confidence intervals for every language.
  • The first nano and lite students distilled from khatraTTS, measured against the teacher on the same sentences.

Get it

There is no model to download. What is public:

  • The per-sentence scores behind the clips and the two charts: khatratts-data.json, with every sample's CER and each clip's CER after voice conversion, and the script that draws the figures from it, make_figures.py.
  • The students and the distillation recipe: sanoTTS.

Credits: the reference voice for English and the other languages is speaker p244 of the VCTK corpus (CC BY 4.0). The Nepali reference voice is a Nepali test voice from the AI4Bharat Rasa dataset (CC BY 4.0). The clips are voice-converted to the voice ne_rvc_f01 of our Nepali RVC voice pack (CC BY 4.0), which was trained with Applio RVC on audio from IndicVoices-R (CC BY 4.0). The test sentences are from FLEURS (CC BY 4.0). The FlowLM code builds on MIT-licensed open-source code from Kyutai's Pocket TTS.

This research model stays internal. The teacher we are building on open data will be released openly.