When do low temperature and high temperature help?

Category: LLM & Prompt Engineering

Answer

Low temperature (0-0.3) narrows the distribution to the top tokens; deterministic, repeatable, repeatable evals. High temperature (0.7-1.2) broadens; diverse, creative, exploratory. Code generation, math, classification -> low temp. Brainstorm, fiction, ideation -> high temp.

Concrete examples from the fca project context

Example 1

temperature=0 + top_p=1 -> greedy; same input always returns same output.

Example 2

temperature=0.7 + top_p=0.9 -> creative but coherent (default for chat).

Example 3

Eval scripts MUST pin temperature to make runs comparable.

Example 4

When SAMPLING: keep top_p small even at temperature>0; do not let the tail in.

Data flow / flow chart

task
  deterministic  -> T=0, p=0.1
  creative      -> T=0.7, p=0.9
  eval scripts  -> T=0 strictly

Takeaway

Low T for truth, high T for variety. Eval scripts always pin T=0.