Skip to Content

Chain-of-Thought Reasoning

When to use

Force the model to show its work — explicitly lay out reasoning steps before the final answer.

Analogy

Asking a student to show math work rather than just write the final answer; partial credit relies on the reasoning path.

Data-flow diagram

   Prompt: 'Solve 2+2*2. Think step-by-step; give the answer last.'
   Reasoning: multiply first; 4; then 2 + 4.
   Answer: 6.

Deep explanation

Chain-of-thought (Wei 2022) is the simplest prompt trick that materially improves multi-step reasoning accuracy. Zero-shot CoT: append ‘lets think step by step’ and the model lays out steps. Few-shot CoT: include worked examples. Costs are extra output tokens. Some providers expose a reasoning parameter that allocates hidden tokens.

Examples

Example 1

prompt = 'A store has 3 boxes, 8 apples each. If 5 apples are eaten, how many remain? Think step by step.'
resp = openai.chat.completions.create(
    model='gpt-4o-mini', messages=[{'role':'user','content':prompt}])

Zero-shot step-by-step trigger lifts accuracy on arithmetic and logic by 10-30 percent.

Example 2

prompt = 'Classify sentiment. Think out loud about WHY before producing final label.'
resp = openai.chat.completions.create(
    model='gpt-4o-mini', messages=[{'role':'user','content':prompt}])

Requiring a WHY before label shifts the model out of impulsive classifications.

Example 3

system = "Structure response as `Reasoning: <steps>; Answer: <final>.`"
resp = openai.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role':'system','content':system},
              {'role':'user','content':'What is 17 percent of 230?'}])

Tagging output (Reasoning, Answer) makes downstream parsing trivial.

Common mistake

Encouraging CoT on tasks that do not need it — slower, more tokens, no accuracy win. Always A/B test before deploying CoT.

Key takeaway

Add CoT for multi-step reasoning ONLY after you have measured the lift. Use zero-shot trigger for one-shot experiments; few-shot CoT for production stability.

Production Failure Playbook

Failure scenario 1: cot-everywhere

Failure scenario 2: trusting-cot-reasoning