Language router
A language router reads a message and answers which language it is written in, so the message can go to the desk that can read it. The one you train here is a language detector for a fixed set of four languages, English, French, German and Spanish, and it answers with one of those four every time, because they are the only answers it knows. Guessing at random would be right 25 percent of the time.
For detecting any language, dedicated tools already exist: fastText's language identification models, for example, "can recognize 176 languages." What this page offers instead is the chance to watch a detector learn, see how it does on each language, and understand why it works.
Training
Training is a guessing game, played tens of thousands of times. The model reads a message and guesses its language. Then it is shown the real one, and every one of its internal numbers, its parameters, is nudged slightly toward the answer that would have been right. Each step covers 32 messages and a run is 1,500 steps, so it makes 48,000 guesses over 1,680 training messages, seeing each one about 29 times.
Only the answer is graded: the model reads the whole message but is marked only on the language that follows it. It trains on your computer's graphics chip when your browser supports WebGPU, and on the processor when it does not.
Letter patterns
The model is not looking words up in a dictionary. Reading one character at a time, it learns the shape of each language: its common endings, its small function words, the way its letters cluster. The idea is older than neural networks. In 1994 William Cavnar and John Trenkle showed that counting short runs of characters is enough to tell languages apart, reporting in one test "a 99.8% correct classification rate on Usenet newsgroup articles written in different languages." Google's CLD3, a detector designed to run in the Chrome browser, also works from runs of characters.
Held-out score
Before training starts, 480 messages, 120 in each language, are set aside, and the model never trains on them. In the studio it is scored on 240 of them, a shuffled sample that includes every language; our own measurement below used all 480. As with every skill here, the set-aside messages were checked for near copies of training messages, so a lightly reworded twin cannot slip in as a free point.
On our own engine, with exactly this setup, it scores 92 percent (measured: 440 of 480 held-out messages, 95% confidence 89 to 94, on a four-way choice where guessing scores 25; German 96, English 92, French 90, Spanish 89).
MASSIVE
The messages come from MASSIVE, a dataset Amazon released in 2022 of more than a million short requests to a voice assistant, translated by professional translators into 51 languages and shared under the CC BY 4.0 license. Translating and checking text for AI is still paid work for professionals, and AITraining.jobs' roles for translators and linguists, from the same company as Nanotech, list the open projects and their pay. Its English text comes from an earlier dataset, SLURP, which is credited here too. Because every language in MASSIVE carries translations of the same requests, the four languages here cover the same topics, so the model has to learn the languages themselves rather than what people happen to talk about in each. Texts that are identical in two languages, which a translated dataset naturally contains, were removed, since nothing could tell them apart. Nanotech uses 540 per language, 420 to train on and 120 held out.
Your own languages
Need Italian, Dutch or Portuguese, or just two of these four? Train a text classifier on your own examples: it takes up to 8 labels, so a router for your own set of languages is one paste away, as long as you have examples of each.
Questions
Why is language detection unreliable on short text?
Because a short text carries few clues. A detector weighs letter patterns, and one or two words may contain none that are distinctive: "no" is a word in English and in Spanish. Cavnar and Trenkle noticed this in 1994 too, writing that their method "works a little better for longer articles." A sentence or so gives this model enough to work with; a single word often does not.
Can it tell languages apart without accents?
It does not rely on accents alone: it learns whole letter patterns, not just the accented letters. How well it copes when the accents are typed away has not been measured, so try it once it is trained: type a French or Spanish sentence without them and see what it says.
What happens with a language it does not know?
It still answers one of its four, because those are the only answers it has; there is no "none of these". To cover other languages, train your own router with examples of each, or use a general tool such as fastText for detecting any language.
Is it free?
Yes. It trains right in your browser, so there is no server doing the work and nothing to meter, and you do not need an account. What you type to test it is read by the model in your browser and sent nowhere.