How an LLM "Reads": Embeddings and the Map of Meaning
LLMs don't read words, they read coordinates. Understand what an embedding is and why every word in your prompt moves the model's focus toward a different "neighborhood" of meaning.
Here's an idea that changes how you write prompts: LLMs don't read words, they read coordinates. Every term you type maps to a location in a mathematical space called an embedding, which captures its meaning and links it to nearby words.
What is an embedding?
An embedding (or vector representation) is a sequence of numbers that places a word in a multidimensional mathematical space. Its position reflects the word's meaning and its closeness to similar words.
Picture a giant library where each book is shelved by topic: each word gets a precise "address", like a linguistic GPS. Machines only understand numbers, so these vectors act as the DNA of a word.
dog → [0.2, -0.5, 0.8, 0.3]
cat → [0.21, -0.48, 0.79, 0.31]The letters differ, but the numbers are almost identical because both are domestic mammals and animals, so they sit very close together.
Closeness = similar meaning
In this space, proximity equals semantic similarity: king ↔ queen, hospital ↔ doctor, blue ↔ red. What feels like common sense to us is pure geometry to an LLM.
You can even do arithmetic on concepts:
king − man + woman ≈ queenThe model captures complex relationships through addition and subtraction of coordinates.
How many dimensions?
Humans picture 3 dimensions; LLMs use hundreds or thousands (for example 768, 1024, 1536, 2048 or more, depending on the model). Each dimension is a distinct feature the model uses to classify meaning, and they're impossible to visualize directly.
A fun tool to explore is the Embedding Projector. Search for "queen" and you'll see nearby words such as Elizabeth, Anne, King, Mary, Princess and Catherine. That closeness comes from training on open literature, where Queen Elizabeth appears thousands of times alongside "queen".
LLM vs. predictive keyboard
| Predictive keyboard (e.g. WhatsApp) | LLM |
|---|---|
| Only guesses the next word after the current one | Moves its attention toward a region of meaning |
| Doesn't search for meaning or understand intent | Builds a response coherent with the context |
| Can't write a full email | Can draft complete, contextual text |
For example, typing "the sky is" shifts the LLM's attention to the region of the vector space related to weather, and it builds a coherent answer from there.
What this means for your prompts
Each word you type acts like a command that moves the model's focus toward a specific "neighborhood" of meanings. Changing one word changes the context, and with it, the quality of the response.
- Choose precise words, not generic ones, to point at the right semantic neighborhood.
- Write in your native language whenever the model handles it well: in a language you don't master you lose cultural nuance and subtle shades of meaning, and those details are what activate the right embeddings.
- Synonyms are close but not interchangeable: each carries different connotations and shifts the response differently.
- Treat every term as a steering input: wording is not cosmetic, it directs the model's attention.
Quick experiment
Test the king − man + woman operation yourself with an embeddings demo or the Embedding Projector, and note what result you get.
Key takeaways
- An embedding is a set of numeric coordinates that encodes a word's meaning.
- Close in space = similar in meaning.
- Every word in your prompt steers the model's attention toward a specific region.
- Use precise words and write in your native language when you can.