Linguabase

The Linguabase is our atlas of English—a dozen or so data tables that together map the language: associations, senses, antonyms, word families, definitions, clues. It looks at English holistically: how common and familiar each word is, what people associate it with, and what it means. We built it over fifteen years to power our word games; in 2026 we released the whole thing into the public domain at linguabase.org.

When a player sees "hiking," they might emphasize nature (scenery, trails) or exercise (fitness, exertion)—same word, different contextual flavors. A good word cloud represents the whole word, not just one angle.

Building this kind of database by hand would be absurdly expensive. The first 10 associations for any word are easy (apple → fruit, red, tree). The next 10 are harder. The next 10 are harder still. This nonlinear difficulty means a skilled lexicographer using modern corpus tools might spend an hour per word to build out 50 associations. Multiply that by 400,000 words and you get 200 person-years of work. At $50/hour, even with a team, you'd spend over $20 million coordinating quality. This is why nobody has done it.

The released dataset is thirteen tables in five groups. Words and scoring: the ranked vocabulary, plus per-word signals like concreteness, game suitability, and a G–X content rating. How words relate: associations (roughly 100 ranked related terms per word), senses, and word families. Opposites: antonyms organized by their axis of opposition, with curated opposite pairs. How words read: about 1.5 million definitions, independently authored rather than drawn from any dictionary, plus short clues from 26 angles. Topics: 72,800 curated categories and 373,000 topic entries. Every table ships in two tiers—the full 2-million-word set, and a curated 400,000-word core, the recommended working threshold for most uses. The dataset is far larger than commercial publishers like Oxford; the careful comparisons live on the Linguabase site (vs. Oxford, vs. free resources, vs. LLMs).

Two layers reach beyond games. The familiarity ranking scores every term by how familiar it is to English speakers—a proxy for difficulty and for how people perceive words—which makes real personalization possible in educational tools and word games. And the category and topic tables are built for grouping-style play: raw material for a new generation of word categorization games.

Most words can be associated in multiple directions. For example, if a word embodies three meanings, the Linguabase expands each sense separately, then interleaves them:

"Space" could go toward astronomy (cosmos, galaxy, astronaut), physical room (parking, storage, breathing room), typography (kerning, leading, margin), or abstract expanse (void, infinity, emptiness). A player might think of any of these—so the database needs to cover all the directions, not just the obvious one.

Why can't you just ask ChatGPT?

You might think you could just ask ChatGPT. But LLMs give you what people typically want—ask for pets, you get cats, dogs, and hamsters. That's by design, and usually it's right. But puzzles need variety across meanings, not the most likely answer. The good news for puzzlemaking is that LLMs are much better at judging variety than producing it. Here's what generation looks like:

These lists are valid but banal. "Espresso, caffeine, brew, aroma"—all one sense of coffee. Where's morning? Tea? Bitter? Wake up?

What is a "good" word association?

Thesauruses give you synonyms, but synonyms aren't what we need. "Happy" has synonyms (joyful, glad). But "apple" doesn't—there's no other word that means apple. For nouns especially, we need free association: what comes to mind when you think "apple"? That's different from synonyms. And it raises a hard question: how do you even define "related"?

There's no correct answer. Rhinoceros, savanna, and mammoth are all related to elephant—but in different ways (similar animal, habitat, evolutionary cousin). Even without a single right answer, though, you can tell a good word cloud from a bad one. The elephant cloud should probably include all three directions.

Here's a harder question:

How We Built It

The Linguabase combines two ingredients: human-written associations (especially for common words) and LLM-assisted curation from more than 130 million validation calls.

We started in 2010 by pulling from existing resources: Roget's Thesaurus, Princeton's WordNet (117,000 synonym sets), Wiktionary, and specialized databases like the NASA Thesaurus. This gave us good coverage for common words like "earth" or "courage." But huge gaps remained in the vocabulary.

In 2013, we tried to fill those gaps with automation. We received a grant of 2.3 million supercomputer hours from NSF's XSEDE program and ran statistical text analysis—the same technique Netflix used for "movies you might like" (latent Dirichlet allocation). We analyzed which words appear together in large text corpora. This produced useful candidates, but also a lot of noise: misspellings from source texts, compound words split incorrectly, uppercase/lowercase confusion. Statistical methods have inherent limits.

We shipped our first game with a limited vocabulary—it was all we could build with pre-LLM methods. When LLMs arrived, we could finally scale up. That enabled our second game, which gives players more freedom and draws from a much larger word space.

But give an LLM a list and ask it to judge, and it works. It can compare items directly and recognize which associations span different senses.

So we use LLMs as editors, not generators. We gather a large pool of candidate words from various sources, give the pool to an LLM along with information about the target word's different meanings, and ask it to sort and rank. This works well. Example: give an LLM the words "walk, stomp, stride, February, April, month" and ask it to sort them into "march" (walking) vs. "March" (the month)—it does this perfectly.

The process has three phases: expand (gather lots of candidate words), audit (have LLMs evaluate them), contract (keep only the good ones). We repeat this. The ranking uses LLM judgments plus other signals:

Generating candidates

• Professional curation: Our lexicographer Orin Hargraves and graduate students hand-built 5,000 curated lists (like "types of sushi" or "architectural styles")
• Reference works: WordNet, Wiktionary, NASA Thesaurus, Getty Art & Architecture Thesaurus, medical terminology databases, 70+ specialized sources
• Library of Congress subject headings: Since 1897, LOC catalogers have organized 17 million books into 648,460 subject categories. Each category becomes a seed for LLM expansion—"18th-century colonial architecture" yields domain-specific words that pure generation misses
• Cross-referencing: If word A lists word B as related, we check whether B should list A

The Library of Congress approach shows our philosophy. We're not asking LLMs to create from nothing. We're leveraging chains of human work: authors wrote books, LOC catalogers organized those books into subjects, we use those subjects as anchors for LLM expansion. The chain: Authors → Books → LOC Catalogers → Subject Headings → LLM Expansion → Word Lists. The LLM's job is to fill in a neighborhood that humans already defined.

Some associations require human curation. LLMs have cultural biases—ask for "breakfast foods" and you get American breakfast: bacon, eggs, pancakes. You don't get shakshuka (Middle East), congee (China), dosa (India), or huevos rancheros (Mexico). These require human knowledge to add.

huevos rancheros

shakshuka

dosa

congee

Some words are actually different words depending on capitalization. "march" (walking) vs. "March" (the month). "swift" (fast) vs. "Swift" (programming language) vs. "SWIFT" (international banking network). We've identified 3,509 English words with two capitalization variants and 86 words with three—cases like swat/Swat/SWAT (to hit, a Pakistani valley, a police unit). LLMs can sort these correctly if you prompt them carefully—the challenge is knowing which words need checking in the first place.

Some words connect to everything. "Mammal" could appear in the word cloud for dog, cat, whale, elephant—every mammal. If we didn't penalize these superconnectors, they'd crowd out more interesting associations. So we downweight words that have too many inbound links, making room for distinctive connections.

Ranking and mixing

• LLM validation: 130 million API calls, focused on judging (not generating)—checking if relationships are real, separating different word senses, flagging bad connections
• Morphology filtering: Removing duplicates like run/runs/running
• Word uniqueness: Penalizing overrepresented words (the superconnector problem above)

Test your intuition about what words are over-linked:

Balanced word clouds

For any word, we want the associations to feel right and play right, and cover all the angles a player might have in mind. The challenge is that words carry multiple meanings for different reasons.

Homographs are unrelated words that happen to share spelling—"pupil" the student has nothing to do with "pupil" the eye part. Polysemy is one word whose meaning branched over time—"mouth" started as the body part, then extended to river mouths and cave mouths, all variations on "opening." Facets are the most common need in English, different aspects of the same meaning—"elephant" is one thing, but you might think of its anatomy, its habitat, its cultural symbolism, or its behavioral traits. Our algorithm expands each sense separately, then interleaves them, maximizing the chance players find what they're looking for regardless of which angle they had in mind.

Here are real examples from the Linguabase—the top 15 associations for selected words:

Most English words connect in 7 hops

We analyzed the network structure of the Linguabase and found a "small world" effect: 76% of random English word pairs connect in seven hops or fewer. The average path length is just 6.43 steps.

What do these paths actually look like?

This is why word association games work. Players feel like they can get from any word to any other—and they're right. The paths exist and they're short. "Batman" to "inspect," "sugar" to "peace"—these seem like stretches, but real paths connect them. English meaning is more tightly connected than people realize.

More Small World findings →

Six demos, with recipes

Data tables alone don’t get used—people need to see what the tables can do. So the centerpiece of linguabase.org is a set of six walkthrough demos: the same word data driving completely different game mechanics. Each page plays first, then derives its numbers from the published tables, step by step, ending in a documented build script you can download, plus a method file written to hand to an LLM assistant.

The six: a clue game (4 Clues), where hints from different angles cross on a single answer; a spelling game (Six Secrets), with a staged anagram builder that yields 1,510 puzzles—4.1 years of dailies—from one spec; two categories games (In a Nutshell and Word Tumble) built from the topic tables; semantic word golf, the mechanic In Other Words plays at full scale, with 41.9 million par-3 holes measured in the data; a dictionary assembled entry by entry from the tables; and HANTAI, an opposites game grown from the antonyms table. The counts are measured with one puzzle spec against one release—change the spec and the supply changes.

Alongside the demos, short illustrated essays work through questions the data can answer: How many words are in English? · How far apart are any two words? · Why words with spaces belong in a vocabulary · Language data and copyright

Research Timeline

• 2010–2016: Lexicographic foundations—curated word lists, reference works, early language data projects
• 2013–2014: NSF supercomputer grant: 2.3 million compute hours of foundational computational linguistics
• 2017: Started building the word association database, combining algorithmic methods with human curation
• 2023–2026: LLM-assisted evaluation at scale—using LLMs to audit for errors, inappropriate content, and imbalanced word senses
• June 2026: Dataset released to the public domain, mirrored on Hugging Face, Zenodo, and Kaggle
• August 2026: Walkthrough demos and build recipes launched at linguabase.org

Funding

We funded this ourselves across fifteen years of development, with help from grants and donated computing resources:

Get the data

The entire dataset is dedicated to the public domain under CC0—commercial use welcome, no permission needed. The data ships as plain files (TSV and Parquet): vocabulary, definitions, associations, senses, antonyms, word families, categories, clues, and topics, ready for offline use in your app.

Everything is human-curated, LLM-validated, and tested in shipping games. The full release is mirrored on three platforms, all carrying the same files; if you use the data in research, cite the Zenodo DOI:

• Hugging Face — load straight into Python with datasets or pandas; browse it in the in-page dataset viewer.
• Zenodo — a versioned, citable archive with a permanent DOI (10.5281/zenodo.20788428).
• Kaggle — explore and prototype in hosted notebooks, or pull the files into your own pipeline.

Work with us

The data is self-serve, and the demos show the recipes. If you want help beyond that—custom exports filtered by vocabulary size, familiarity band, or domain; guidance on pathfinding, difficulty scoring, and content filtering; or puzzle content generated for your game mechanics—get in touch.