Temperature and Top P: Why the Same Prompt Gives Different Results
Two hidden settings explain why the same prompt doesn't always answer the same way. Learn what temperature and Top P do and when to raise or lower them.
Ever brewed coffee with the same recipe and gotten a different result each day? LLMs do something similar: the same prompt can produce different results because of two hidden settings, temperature and Top P. Both regulate the model's "creativity", but they work differently. You'll find them in the playground of tools like Google AI Studio (Top P is under advanced settings).
How the model generates text
LLMs predict the next word (token) one step at a time, using the attention mechanism and the available context window. At each step the model has a probability distribution over possible next tokens. Temperature and Top P change how the model picks from it.
Temperature: the creativity thermometer
It controls how random the token selection is.
- Low temperature (e.g. 0.05): consistent, predictable, safe output. Same format and structure, stable information, and it follows instructions strictly (e.g. respects length limits).
- High temperature (e.g. 1): more varied, surprising, elaborated output, but it takes liberties with instructions.
The experiment: Little Red Riding Hood
Generate a story similar to Little Red Riding Hood, your own version, 250 characters.| Setting | Result |
|---|---|
| Temperature 0.05 | A minimal story: Green Riding Hood brings a cake to her grandmother, a fox tricks her, a woodcutter saves them. It respected the limit |
| Temperature 1 | A richer story: an "intrepid" girl, golden apples, the wolf challenged with a riddle, a happy ending. It broke the character limit |
When to use each level
| Task type | Setting | Examples |
|---|---|---|
| Precision and consistency | Low | Legal document analysis, financial reports |
| Creative work | High | Video titles, social media posts, storytelling |
| Mixed cases | Medium (experiment) | Social post based on a scientific article |
Top P (nucleus sampling)
It controls the pool of words the model may choose from when building a response.
- Low Top P: only the most probable, safest words, so less variety and more repetition.
- High Top P: a wider vocabulary, with more synonyms and more sophisticated, diverse language.
Technical detail: Top P keeps the smallest set of tokens whose cumulative probability reaches P. For example, Top P = 0.1 keeps only the top candidates covering 10% of the probability mass.
Experiment results: with Top P 0.05, the story was nearly identical to the low-temperature one. With Top P 1, the model invented a character (Luna), added dialogue, and wrote a richer narrative.
Temperature vs. Top P
| Temperature | Top P | |
|---|---|---|
| Controls | How random the choice is | Which words are eligible |
| Low value | Predictable, consistent | Safe, repetitive vocabulary |
| High value | Surprising, elaborate | Diverse, sophisticated vocabulary |
Another technical detail: temperature reshapes the probability distribution (flatter or sharper), while Top P truncates it.
Can you combine them?
Many guides recommend changing only one at a time. In practice, combining them can work well depending on the problem. With both at 0.05 you get the most conservative result (short, predictable, no surprises).
- Reports and analysis: lower both for maximum precision.
- Creative work: raise both, then adjust to taste.
- Mixed problems: experiment with combinations until you find the balance.
A practical workflow: Google AI Studio lets you save each version and compare them in the history, so it works as a testing lab. Before choosing settings, ask yourself whether you need accuracy or creativity, and test on a real problem.
Key takeaways
- Different results from the same prompt are not a bug; they come from sampling settings.
- Low values = control and consistency. High values = creativity and variety, with less instruction adherence.
- Temperature adjusts randomness; Top P adjusts the vocabulary pool.
- There is no universal best setting: it depends on the task, so iterate and compare.