Building Swahili Speech on Dia-1.6B

We're releasing an open Swahili text-to-speech model based on Dia-1.6B.

The model produces natural Swahili speech from text, supports Swahili-English code-switching, and is released under the CC-BY-4.0 license. It was trained entirely on openly licensed speech datasets and is accompanied by the training pipeline, evaluation harness, benchmark, and model card.

This project began with a different conclusion.

When we first evaluated Dia as a foundation for Swahili speech synthesis, we decided against using it. The model was English-only, there was no official fine-tuning pipeline, and we could not find a documented example of anyone successfully adapting it to a new language. Our own initial experiments also failed, producing a character error rate (CER) of 2.331.

We revisited that conclusion after changing both our training approach and our evaluation methodology.

The resulting model was trained on 126 hours of Swahili speech and reaches 0.011 CER on our curated evaluation set.

More importantly, the process exposed several assumptions in our tooling that turned out to be incorrect. Those findings are as valuable as the model itself, so we're releasing both.

Model capabilities

The model synthesizes a single natural voice at 44.1 kHz from text input.

It does not require reference audio or speaker conditioning.

One area where it performs particularly well is Swahili-English code-switching. Mixed-language sentences such as:

Nitakutumia link ya meeting kwenye email yako.

are common in conversational Swahili but remain challenging for many text-to-speech systems. During evaluation, this model proved substantially more robust on these transitions than the alternatives we tested.

Evaluation

We report both single-shot generation and generation with ASR-gated retries.

The benchmark consists of 500 previously unseen Swahili news sentences. On a smaller curated evaluation set, the retry configuration reaches 0.011 CER.

Reporting both numbers is important because they measure different deployment scenarios. Single-shot generation reflects what a user receives from one synthesis pass. Retry-based evaluation measures the quality obtainable when generation can be repeated automatically until transcription agrees with the input.

Most of the improvement comes from rare named entities rather than ordinary conversational text, suggesting that the model's uncertainty is concentrated in infrequent lexical items rather than general pronunciation.

Lessons from evaluation

One of the most important findings from this project concerns evaluation rather than synthesis.

Our initial model selection relied on a Swahili automatic speech recognition system. That benchmark consistently preferred one checkpoint over another.

When the same checkpoints were evaluated by a native Swahili speaker, the ordering reversed.

The explanation was straightforward.

A Swahili-only recognizer cannot accurately transcribe embedded English words. Errors on words such as link and meeting reflected limitations of the evaluator rather than the synthesized speech itself.

Replacing the evaluator with a multilingual ASR system produced rankings that matched human judgment.

This changed the checkpoint we ultimately released.

The broader lesson is that evaluation systems inherit the limitations of the models used to construct them.

...