Large Language Models are Zero-Shot Reasoners
- Publication
- NeurIPS 2022Conference
- First submitted to arXiv
- arXiv:2205.11916
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{transformerszeroshotcot2022,
title = {{Large Language Models are Zero-Shot Reasoners}},
author = {Takeshi Kojima and Shixiang Shane Gu and Machel Reid and Yutaka Matsuo and Yusuke Iwasawa},
year = {2022},
eprint = {2205.11916},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2205.11916}
}
The GPT-3 paper was titled "Language Models are Few-Shot Learners." Ever since, when people talk about what makes large language models impressive, the first thing that comes to mind is few-shot learning: put a few examples in the prompt, and the model picks up a new task.
Chain-of-thought prompting, from January 2022, followed the same path. Write the reasoning steps into the exemplars' answers, the model writes its own the same way, and math scores jump. That paper didn't even report zero-shot results, as if without exemplars there was nothing worth comparing.
Four months later, Kojima of the University of Tokyo, Shane Gu of Google and colleagues found that the exemplars can go entirely. Just add the sentence "Let's think step by step" before the answer, and text-davinci-002 rises from 17.7% to 78.7% on MultiArith and from 10.4% to 40.7% on GSM8K.
Defining the problem
Few-shot chain of thought has two problems. People have to write the exemplars by hand, one task at a time; the reasoning steps have to be well written, and the answer format has to be consistent. And when the exemplars don't match the questions, performance drops noticeably. We'll see the numbers later.
Zero-shot prompting had been tried before, but mostly by designing a separate template for each task, so every new task meant thinking one up again.
What this paper asks is whether there's a single sentence, unrelated to any particular task, that gets the model to write multi-step reasoning without any exemplars at all.
Solving the problem

The top row is few-shot and the bottom row zero-shot; the left side answers directly and the right side writes reasoning first. The bottom right is this paper's method. It needs no exemplars, and every task uses the same sentence.
Two-stage prompting
![Original Figure 2. The full zero-shot chain-of-thought pipeline. The first prompt is "Q: [question] A: Let's think step by step.", and the language model continues with the reasoning: 25 punches a minute, 75 punches in three minutes, 375 punches in five rounds. The second prompt joins the question, the first-stage reasoning and "Therefore, the answer (arabic numerals) is", and sends it into the same language model, which gives 375.](/content/papers/assets/f07eb505c9d1e936a0672d90c6569c69aedf4530432157e51976e887ecef11be.png)
The method fits in one sentence, but in practice you have to ask the model twice:
- Reasoning extraction: put the question into "Q: [question]. A: Let's think step by step." and let the model continue, which gives the reasoning . The whole paper uses greedy decoding.
- Answer extraction: take the first-stage prompt and the reasoning , append an answer trigger, and send it back to the same model. Numeric questions use "Therefore, the answer (arabic numerals) is", and multiple-choice questions use "Therefore, among A through E, the answer is".
The second stage is needed because there's no exemplar demonstrating "end with The answer is X," so after writing its reasoning the model won't necessarily behave and give an answer that can be parsed. Few-shot prompting doesn't need this step, because the exemplars have already taught the answer format.
Once the model has given its answer, there's one more cleanup pass. For numeric questions take the first number that appears, for multiple choice the first capital letter, and for yes/no questions the first "yes" or "no." In the boxing problem above, the model writes "375." in the second stage, which cleans up to 375. These rules are crude. If the model writes "B, C, and D" in one go, only B gets picked up, and when the sentence starts with "Among," the first capital letter caught is the A in "Among." Still, every method uses the same rules, so the comparison is reasonably fair.
Experimental setup
- Tasks: 12 datasets in four categories. Arithmetic: SingleEq, AddSub, MultiArith, AQUA-RAT, GSM8K and SVAMP, where the first two can mostly be solved in a single step. Commonsense: CommonsenseQA and StrategyQA. Symbolic reasoning: last-letter concatenation over four names, and a coin flipped four times. Other logical reasoning: date understanding and tracking three shuffled objects, both from BIG-bench.
- Models: 17 in all, from 0.3B to 540B. The main ones are InstructGPT (text-ada/babbage/curie/davinci-001 and text-davinci-002), the original GPT-3 in four sizes, and PaLM 8B/62B/540B; the scaling experiments add GPT-2, GPT-Neo, GPT-J, T0 and OPT. Unless stated otherwise, the model is text-davinci-002.
- Baselines: the zero-shot baseline also uses similar answer triggers; few-shot and few-shot chain of thought use the exemplars from Wei et al., with the same set of 8 for the arithmetic problems.
Discussion
Against zero-shot
| Dataset | Zero-shot | Zero-shot CoT |
|---|---|---|
| SingleEq | 74.6 | 78.0 |
| AddSub | 72.2 | 69.6 |
| MultiArith | 17.7 | 78.7 |
| GSM8K | 10.4 | 40.7 |
| AQUA-RAT | 22.4 | 33.5 |
| SVAMP | 58.8 | 62.1 |
| CommonsenseQA | 68.8 | 64.6 |
| StrategyQA | 12.7 | 54.8 |
| Date understanding | 49.3 | 67.5 |
| Shuffled objects | 31.3 | 52.4 |
| Last letters | 0.2 | 57.6 |
| Coin flip | 12.8 | 91.4 |
Compiled from Table 1 of the paper, using triggers designed for each answer format; values are in %.
Problems that need multi-step reasoning all improve a lot. MultiArith gains 61 percentage points, last-letter concatenation goes from almost 0 to 57.6%, and coin flipping from 12.8% to 91.4%. SingleEq and AddSub never really needed multi-step calculation, and they stay about the same.
CommonsenseQA actually gets worse, dropping from 68.8% to 64.6%. The authors point out that the chain-of-thought paper saw something similar. Few-shot chain of thought didn't help on commonsense questions with LaMDA 137B, and only worked on StrategyQA once it got to PaLM 540B.
Two numbers in the table deserve a second look. StrategyQA and coin flipping are both yes/no questions, where random guessing gets 50%, yet the zero-shot baseline only gets around 12%. The original table also reports results using "The answer is" to extract the answer, and with that, the two zero-shot baselines come back to 54.3% and 53.8%. It looks like part of the zero-shot baseline's low score comes from answers not being extracted properly; for zero-shot chain of thought, the gap between the two extraction methods is much smaller.
Against few-shot
| Method | MultiArith | GSM8K |
|---|---|---|
| Zero-shot | 17.7 | 10.4 |
| Few-shot (8 exemplars) | 33.8 | 15.6 |
| Zero-shot CoT | 78.7 | 40.7 |
| Few-shot CoT (2 exemplars) | 84.8 | 41.3 |
| Few-shot CoT (8 exemplars) | 93.0 | 48.7 |
| CoT with 8 exemplars, plus "Let's think step by step." before the answer | 92.8 | 51.5 |
| Fine-tuned GPT-3 175B | - | 33 |
| Fine-tuned GPT-3 175B + verifier | - | 55 |
| PaLM 540B: zero-shot | 25.5 | 12.5 |
| PaLM 540B: zero-shot CoT | 66.1 | 43.0 |
| PaLM 540B: zero-shot CoT + Self-Consistency | 89.0 | 70.1 |
| PaLM 540B: few-shot CoT | - | 56.9 |
Compiled from Table 2 of the paper, showing selected rows; rows without a model name are text-davinci-002, and values are in %.
Zero-shot chain of thought naturally can't match few-shot chain of thought with carefully written exemplars. But it beats standard few-shot prompting with 8 exemplars, and on GSM8K it beats the fine-tuned GPT-3 175B. In other words, the gain those 8 exemplars bring to few-shot prompting is far smaller than what one line saying "think step by step first" brings.
Add this sentence to the few-shot chain-of-thought exemplars, and GSM8K rises further, from 48.7% to 51.5%. With PaLM 540B plus Self-Consistency's sampling and voting, even zero-shot reaches 70.1%.
The model has to be big enough

Without reasoning, scores barely move as the model grows; with this sentence added, scores climb steeply with model size. On small models it does nothing, and the reasoning they write is poor too. This agrees with the few-shot chain-of-thought results. Reasoning ability only shows up at a certain scale, and this sentence just draws it out.
The gain on the original GPT-3 175B is far smaller than on the instruction-tuned text-davinci-002. In their discussion, the authors admit that none of these models' training-data details are public, so they can't be sure where the difference comes from.
Would a different sentence work?
| Category | Trigger | MultiArith |
|---|---|---|
| Instructive | Let's think step by step. | 78.7 |
| Instructive | First, | 77.3 |
| Instructive | Let's think about this logically. | 74.5 |
| Instructive | Let's solve this problem by splitting it into steps. | 72.2 |
| Instructive | Let's think like a detective step by step. | 70.3 |
| Instructive | Let's think | 57.5 |
| Instructive | The answer is after the proof. | 45.7 |
| Misleading | Don't think. Just feel. | 18.8 |
| Misleading | Let's think step by step but reach an incorrect answer. | 18.7 |
| Misleading | By using the fact that the earth is round, | 9.3 |
| Irrelevant | By the way, I found a good restaurant nearby. | 17.5 |
| Irrelevant | Abrakadabra! | 15.5 |
| None | (Zero-shot) | 17.7 |
Compiled from Table 4 of the paper, 12 of the 16 triggers; text-davinci-002, values are in %.
Any sentence that steers the model toward reasoning helps, but the differences are sizable. "Let's think" without "step by step" falls from 78.7% to 57.5%. Misleading and irrelevant sentences all sit near the zero-shot level, and "By using the fact that the earth is round" even drags it down to 9.3%. The authors also observed that different triggers make the model write its reasoning in very different ways. As for how to find better triggers automatically, the authors left that as an open question.
When the exemplars don't match the questions
| Method | AQUA-RAT | MultiArith |
|---|---|---|
| Zero-shot | 22.4 | 17.7 |
| Few-shot CoT, exemplars from CommonsenseQA | 31.9 | 27.0 |
| Zero-shot CoT | 33.5 | 78.7 |
| Few-shot CoT, exemplars from the same task | 39.0 | 88.2 |
Compiled from Table 5 of the paper; values are in %.
Here, exemplars from commonsense questions are used to do math problems. AQUA-RAT and CommonsenseQA are both multiple choice, with the same format, so the exemplars still help a little; MultiArith wants a numeric answer, a different format, and the exemplars are almost useless. This matches the observation by Min et al. that what a model picks up from exemplars is mostly the answer format rather than the task itself. In both cases, the result loses to zero-shot chain of thought.
Where it goes wrong

The authors manually inspected text-davinci-002's outputs:
- Commonsense questions: the reasoning is often sensible, just with an answer that differs from the reference. In Example 1, "the toy car is in the house" isn't wrong. When the model can't choose among several options, it also picks several at once.
- Arithmetic questions: zero-shot chain of thought often already has the right answer, then keeps writing a few more steps and changes it to a wrong one; sometimes it never really starts reasoning and just restates the question. Few-shot chain of thought fails differently, often going wrong on expressions like (3 + 2) * 4 that do two operations in one go.
Limitations
The authors raise two points:
- Opaque training data: the differences between the 001 and 002 GPT-3 models, and what data PaLM used, were never made public. The authors' argument is that the consistent gains across models, and across arithmetic and non-arithmetic tasks, look less like memorized answers and more like a general reasoning ability.
- Bias: the prompt exploits patterns the model learned from web data, so the biases in that data come along with it.
Also, most of the results are on text-davinci-002 with greedy decoding only; the key results were reproduced on PaLM.
What came after
"Let's think step by step" quickly became the most common zero-shot prompt, and the open question the authors left behind was soon picked up by others.
In November 2022, APE from the University of Toronto had a language model propose candidate instructions itself and then scored them. They first used "Let's think step by step" to generate a batch of reasoning and threw away the ones with wrong answers (a step borrowed from STaR), then searched for the opening sentence most likely to produce those correct lines of reasoning, and found "Let's work this out in a step by step way to be sure we have the right answer." On text-davinci-002, MultiArith rose from 78.7% to 82.0%, and GSM8K from 40.7% to 43.0%.
In September 2023, Google DeepMind's OPRO used a language model as an optimizer, repeatedly proposing new sentences, checking their scores and rewriting. With PaLM 2-L answering the questions, the best sentence was "Take a deep breath and work on this problem step-by-step.", at 80.2% on GSM8K; "Let's think step by step" got 71.8%, and the sentence APE had found managed only 58.8%. The best sentence changes with the model. The best one found on text-davinci-002 fell behind once it was moved to PaLM 2-L.
With reasoning models, things came full circle. These models already write long reasoning before they answer, and OpenAI's prompting guide for reasoning models says outright to avoid chain-of-thought prompts, because these models reason internally, so asking them to "think step by step" or "explain your reasoning" is unnecessary.
Conclusion
Add "Let's think step by step" before the answer, and text-davinci-002 goes from 17.7% to 78.7% on MultiArith and from 10.4% to 40.7% on GSM8K, beating standard few-shot prompting with 8 exemplars along the way. That's where this paper carries the most weight. The reasoning ability was in the model all along, and a large part of what few-shot exemplars provide is just format.
But it's also picky. The model has to be big enough, and it doesn't help on commonsense questions. Change a few words in the trigger and the score moves by 20 percentage points; switch models and the best sentence changes with it.
What came next is a little ironic. People spent more than a year using algorithms to find better ways to phrase this sentence. Then reasoning models trained "think before you answer" straight into their weights, and the prompting guides turned around and asked you to stop reminding them.