# Offline voice (Omnilingual)

> One offline model for about 380 keyboard languages, many with no other voice typing: Meta's Omnilingual ASR on your phone.

URL: https://wmkeyboard.pages.dev/tools/omnilingual/
Section: Tools

Offline Omnilingual runs [Omnilingual ASR](https://github.com/facebookresearch/omnilingual-asr), Meta's speech recognition model for more than 1,600 languages, on your phone with ONNX Runtime, the engine [Offline Sherpa](https://wmkeyboard.pages.dev/tools/sherpa/) runs on. One model covers every language it knows, and it works out which one you speak by itself, while the keyboard keeps what it writes to your language's script. About 380 of the keyboard's languages are among them: Hindi, Tamil, Swahili, Yoruba and Persian, but also hundreds that no other engine here can take, such as Nuer, Nande or Tarifit. Once the model is downloaded, no audio leaves the phone.

## Setting it up

Voice typing

1. Add **Omnilingual** to the **Recognition engine** list and move it to the top, or pick it for one language under **Engine for each language**. Picking it per language is the better fit when another engine already serves your main language well.
2. Press **Omnilingual** on the list to open its screen, which holds the **Omnilingual model** group. The first time, a banner offers ONNX Runtime, the engine it runs on, shared with Sherpa: a one-time download of about 33 MB (arm64), from Google Play on the Play build and from Maven Central on the GitHub and F-Droid builds; the offline build carries it already. The F-Droid build asks you to turn on **Download feature libraries** first.
3. Download **Omnilingual ASR 300M**: 367 MB from Hugging Face at a fixed revision, every file checked against its published SHA-256. On mobile data it asks first.

| Model | Languages | Download | Licence |
|---|---|---|---|
| Omnilingual ASR 300M | about 380 keyboard languages (1,600 in all) | 367 MB | Apache-2.0 |

The download includes Silero VAD (2 MB), the voice detector that cuts your speech into phrases.

## How it works

Omnilingual is not a streaming model: it reads a whole phrase at once. While you talk, Silero listens for the short pause at the end of each phrase (half a second) and hands that phrase to the model while you go on to the next one. **Phrase end detector** on the Voice typing screen can put FireRedVAD, plain loudness or a model of your own in Silero's place; see [Hearing the end of a phrase](https://wmkeyboard.pages.dev/tools/voice/#hearing-the-end-of-a-phrase). The words therefore arrive a phrase at a time, as underlined composing text in the field and on the line under the microphone, and not word by word as they do with Sherpa. When the dictation ends, the text goes through the same spacing, spoken punctuation, casing and undo as any other dictation (see [Voice typing](https://wmkeyboard.pages.dev/tools/voice/)).

**When dictation ends.** With **Stop after a pause** set, a longer pause of that length ends the dictation; with **Keep listening** on, the next one starts at once. With the setting off it runs until you press the microphone or two minutes pass. A phrase that runs past 20 seconds without a pause is cut at the next short breath, because the model reads at most 40 seconds at a time.

**Plain words.** The model writes words without punctuation or capitals. With [Add punctuation and capitals](https://wmkeyboard.pages.dev/tools/voice/#adding-punctuation-and-capitals) on, the default, a punctuation model for 88 languages puts in full stops, commas, question marks and sentence capitals once it is downloaded from that setting's row on the Voice typing screen. Without it the keyboard capitalises the start of a dictation, and spoken punctuation ("comma", "full stop") works as usual.

**Memory and speed.** The model takes about 1 GB of memory while it is open, so it suits a recent phone. The keyboard frees it when Android asks for memory back, and opening it closes Sherpa's model first. Measured on a Mac, it needs about 2.1 billion processor cycles per second of speech on one core; with four threads that comes to an estimated 0.3 seconds of waiting per second of speech on a flagship phone and about 0.5 on a mid-range one.

## Omnilingual, Whisper or the others

Use Omnilingual for a language that has no other offline engine here. Where one exists, it is usually the better choice: [Whisper](https://wmkeyboard.pages.dev/tools/whisper/) is more accurate on the languages it was trained hardest on, [Sherpa](https://wmkeyboard.pages.dev/tools/sherpa/) streams its 13 languages word by word, and [Parakeet](https://wmkeyboard.pages.dev/tools/parakeet/) covers 25 European ones. On 12 FLEURS test clips per language, Omnilingual's character error rate was 14.3% for Hindi, 19.4% for Tamil, 5.3% for Swahili and 21.0% for Yoruba.

Meta's larger MMS-1B model was measured too and was more accurate (6.4% for Hindi), but at 1.9 GB of memory and about two and a half times the work per second of speech it is too heavy for a keyboard, so it is not offered.

## Details & edge cases

**No language to choose.** The model has no way to be told the language, so it guesses from the sound. A dictation that listens for several languages at once works the same way, as long as Omnilingual knows all of them.

**Kept to your script.** Left to itself the model spells by sound in whichever script it thinks it hears, so Hindi often comes out in Urdu letters and an English phrase now and then in Arabic ones. The keyboard therefore lets it write only the letters of the script of the language you dictate in, plus digits, spaces and punctuation. Where the model reaches for another script's letter, the letter of your script that it rates best for that sound goes in instead: Urdu ب becomes ब, Arabic ل becomes l. A dictation in several languages may use the scripts of all of them, so Hindi with English keeps both Devanagari and Latin. A phrase the model heard as the neighbouring language can still be spelled the way that language writes it. Urdu leaves out most short vowels, so a Hindi word may now and then miss a vowel sign, and English heard as Arabic loses some vowels the same way. For a language whose script the keyboard has no Unicode name for, or a notation such as music or Braille, the model writes as it likes.

**Scripts.** It writes each language in the script it was trained on. Languages it only writes in a different script than the keyboard's are left out of its list, so they go to another engine: Crimean Tatar (Cyrillic, not Latin), Manipuri (Bengali, not Meetei Mayek), Tulu (Malayalam, not Kannada) and a few others.

**No words to listen for.** The model takes no list of names or terms, so **Words to listen for** does not show while it is the engine.

**The licences.** Omnilingual ASR is Meta's under Apache-2.0, converted for sherpa-onnx by k2-fsa. sherpa-onnx is Apache-2.0, ONNX Runtime is Microsoft's under MIT, and Silero VAD is MIT. All of them are credited in the app's About screen.