Back to paper library
arXiv 2025EMNLP 2025Model adaptationPaper notes

s1

Wait, think again

Original paper

s1: Simple test-time scaling

Paper authors

Niklas Muennighoff · Zitong Yang · Weijia Shi …

Show all 10 authors

Niklas Muennighoff · Zitong Yang · Weijia Shi · Xiang Lisa Li · Li Fei-Fei · Hannaneh Hajishirzi · Luke Zettlemoyer · Percy Liang · Emmanuel Candès · Tatsunori Hashimoto

Publication
EMNLP 2025Conference
First submitted to arXiv
arXiv:2501.19393
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{modeltunings12025,
  title = {{s1: Simple test-time scaling}},
  author = {Niklas Muennighoff and Zitong Yang and Weijia Shi and Xiang Lisa Li and Li Fei-Fei and Hannaneh Hajishirzi and Luke Zettlemoyer and Percy Liang and Emmanuel Candès and Tatsunori Hashimoto},
  year = {2025},
  eprint = {2501.19393},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.19393}
}
On this page

In September 2024, OpenAI released o1 and showed off a lovely curve: the longer the model thinks, the more accurate its answers. But OpenAI didn't disclose how it was done, saying only that it used large-scale reinforcement learning.

Over the following months, many teams tried to reproduce it. Some used Monte Carlo tree search, some had multiple agents debate each other; DeepSeek-R1, in January 2025, used millions of samples and several stages of reinforcement learning to catch up with o1's scores. Yet among all these reproductions, no one had openly reproduced that "think longer, answer better" curve.

Muennighoff, Yang and colleagues from Stanford, the University of Washington, AI2 and elsewhere asked a very direct question: what is the simplest way to get both that curve and strong reasoning ability? Their answer is surprisingly simple. 1,000 examples, 26 minutes of fine-tuning on 16 H100s, plus one "Wait."

Original Figure 1. s1-32B's accuracy against average thinking tokens on three reasoning benchmarks. MATH500 rises from about 65% at about 512 tokens to about 93% at about 2,048 tokens; AIME24 rises from near 0 at about 512 tokens to about 57% at about 8,192 tokens; GPQA Diamond rises from about 40% at about 1,024 tokens to about 60% at about 4,096 tokens. On all three, the longer the thinking, the higher the accuracy.
Original Figure 1, Muennighoff et al., from PDF page 1, arXiv license, scholarly quotation.

Defining the problem

The authors divide methods that spend more compute at inference time into two kinds:

  • Sequential: later computation builds on earlier results, as in writing out a very long chain of reasoning.
  • Parallel: computations run independently and are aggregated at the end, as in majority vote.

The authors focus on sequential methods, on the intuition that later computation can make use of earlier intermediate results, reason more deeply, and revise again and again.

To compare methods, accuracy alone isn't enough; you also need to know whether the compute can be controlled, and whether accuracy rises as compute increases. The authors define three metrics. For a given method, run it several times on the same benchmark with different amounts of compute a∈Aa \in \mathcal{A}, giving accuracy f(a)f(a):

Control=1∣A∣∑a∈A1(amin≤a≤amax),Scaling=1(∣A∣2)∑a,b∈A, b>af(b)−f(a)b−a,Performance=max⁡a∈Af(a)\text{Control} = \frac{1}{|\mathcal{A}|}\sum_{a \in \mathcal{A}} \mathbb{1}(a_{min} \le a \le a_{max}), \qquad \text{Scaling} = \frac{1}{\binom{|\mathcal{A}|}{2}}\sum_{a, b \in \mathcal{A},\, b > a} \frac{f(b) - f(a)}{b - a}, \qquad \text{Performance} = \max_{a \in \mathcal{A}} f(a)

Control is the fraction of runs in which the thinking tokens actually used fall within the specified range; Scaling is the average slope of the curve, which has to be positive to be useful; Performance is the highest accuracy reached.

Solving the problem

s1K: picking 1,000 questions

Original Figure 2. On the left are the domains covered by s1K's 1,000 questions, shown as bubbles sized by count, including geometry, number theory, combinatorics, probability theory, calculus, biology, astronomy, economics and quantum theory, 50 domains in all. On the right is MATH500 accuracy against the number of examples used for reasoning fine-tuning: s1 uses only 1,000 examples and reaches about 93% accuracy, sitting in the top-left "most sample-efficient" region; Bespoke-Stratos and Sky-T1 used 17,000 examples, and r1-distill used 800,000; the example counts for QwQ and o1-preview are undisclosed.
Original Figure 2, Muennighoff et al., from PDF page 3, arXiv license, scholarly quotation.

First, collect 59,000 questions. There are 16 sources; the largest is NuminaMATH, with 30,660 math problems, along with past AIME problems (1983 to 2021), OlympicArena's interdisciplinary olympiad problems (4,250), OmniMath's competition math (4,238), and AGIEval's standardized exams such as the SAT and LSAT (2,385). The authors also made two sets of their own: 182 probability questions from Stanford's statistics PhD qualifying exam, and 23 hard brain teasers of the kind common in quantitative trading interviews.

Every question was run through Google's Gemini 2.0 Flash Thinking to generate a reasoning trace and a solution, giving 59,029 "question, reasoning, solution" triples. Any data sharing a run of 8 consecutive words with the benchmark questions (MATH500, GPQA Diamond, AIME24) was removed, and duplicate questions were dropped as well.

Then three principles are used to filter:

  1. Quality: remove those with API errors, leaving 54,116 questions; then remove those with formatting problems, such as ASCII art, references to nonexistent images, or garbled question numbering, leaving 51,581. Of these, 384 come from high-quality datasets that need no further filtering and go straight into the final list.
  2. Difficulty: have Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct each attempt every question once, with Claude 3.5 Sonnet grading them against the reference answer. If either model gets it right, the question is treated as too easy and dropped, leaving 24,496. The length of the reasoning trace is also used as a measure of difficulty, on the assumption that harder questions need longer thinking.
  3. Diversity: have Claude 3.5 Sonnet assign each question to a domain according to the American Mathematical Society's Mathematics Subject Classification. Each time, first pick a domain at random, then pick a question from that domain, with longer reasoning traces more likely to be picked; repeat until there are 1,000 questions, which end up covering 50 domains.

The grading model judges only 53.6% of the reasoning traces in s1K to be correct. The authors deliberately keep these mistakes; what they want is the reasoning process, not perfectly correct solutions.

Budget forcing

Original Figure 3. An example of budget forcing. The question is "How many r in raspberry?" The reasoning trace checks the letters one by one: the first letter, r, count 1; the seventh letter, r, count 2; the eighth letter, y, count still 2; "The number of 'r's in "raspberry" is 2." The model wants to stop thinking here, but the end-of-thinking token is removed and a red "Wait" is appended instead; the model goes on to write "let's re-read the question carefully," counts again one by one, and finds the third r, count 3. The final response reads "My initial answer of 2 was incorrect due to a quick reading of the word," with a final answer of 3.
Original Figure 3, Muennighoff et al., from PDF page 4, arXiv license, scholarly quotation.

The method for controlling thinking length is called budget forcing, and it intervenes directly during decoding:

  • Setting a maximum: once the number of thinking tokens exceeds the limit, append the end-of-thinking token directly, adding "Final Answer:" if needed, to force the model to hand over its best answer so far.
  • Setting a minimum: when the model tries to stop thinking, swallow the end-of-thinking token and append "Wait" instead, so it keeps thinking.

The figure above is one example. The model counted once, concluded that raspberry has only two r's, and tried to stop; after "Wait" was appended, it reread the question and found the third r.

What the model writes after "Wait" depends on what it learned from its training data. The reasoning traces in s1K are full of sentences like "wait a moment" and "let me check that again," so when the model sees "Wait," it goes on to check its earlier reasoning.

The authors also compare several other ways of controlling length:

  • Token count in the prompt: tell the model the maximum number of tokens to think for.
  • Step count in the prompt: tell the model the maximum number of steps to think for, with each step about 100 tokens.
  • Short or long in the prompt: two prompts, one telling the model to think briefly and one telling it to think longer.
  • Rejection sampling: keep sampling until one generation happens to fall within the budget.

Training

The base model is Qwen2.5-32B-Instruct. The thinking part starts with <|im_start|>think and the answer part with <|im_start|>answer. Training runs on s1K for 5 epochs with batch size 16, 315 steps in total, at learning rate 10−510^{-5}, with warmup over the first 5% followed by cosine decay to 0; the loss is computed only on the reasoning trace and the solution, not on the question.

Training took 26 minutes on 16 H100s, about 7 GPU hours.

Discussion

Think longer, answer better

Original Figure 4. (a) Sequential scaling on AIME24. The points on the left cut thinking off at a maximum (such as 2,048 or 4,096 tokens), with accuracy of about 13% to 43%; the point in the middle at about 50% is no intervention; the three rightmost points append "Wait" 2, 4 and 6 times when the model tries to stop, with accuracy rising to about 57% and then leveling off. (b) A comparison on GPQA Diamond. The dark blue dashed line is s1 scaling sequentially with budget forcing, rising from about 51% to about 60% as output tokens increase; the light blue dashed line is Qwen2.5-32B-Instruct scaling in parallel with majority vote, staying between about 46% and 52% and failing to catch up even at a million total tokens.
Original Figure 4, Muennighoff et al., from PDF page 5, arXiv license, scholarly quotation.

Stretching the thinking with budget forcing lifts AIME24 from 50% with no intervention to 56.7%. But by the sixth "Wait," the curve has gone flat; swallow the end-of-thinking token too often and the model falls into repetitive loops instead of continuing to reason.

The right panel compares the two ways of scaling. The base model, Qwen2.5-32B-Instruct, is sampled 64 times per question at temperature 1, with majority votes over 2 to 64 answers, and its accuracy never catches up with s1's sequential scaling. The authors take this as confirming their initial intuition that sequential scaling is more effective than parallel scaling.

Results

ModelReasoning examplesAIME24MATH500GPQA
o1-previewUndisclosed44.685.573.3
o1Undisclosed74.494.877.3
Gemini 2.0 Flash ThinkingUndisclosed60.0--
Qwen2.5-32B-Instruct-26.784.049.0
DeepSeek-R1Far more than 800K79.897.371.5
R1 distilled (32B)800K72.694.362.1
Bespoke-32B17K63.393.058.1
s1 (no budget forcing)1,00050.092.656.6
s1-32B1,00056.793.059.6

Compiled from Table 1 of the paper, selected rows; in %.

With just 1,000 extra training examples, s1-32B scores 30 percentage points higher than the base model on AIME24. On both AIME24 and MATH500 it beats o1-preview; the abstract's claim of being "up to 27%" higher is the relative gap of 56.7 versus 44.6 on AIME24. On GPQA, though, it's still a dozen or so percentage points below o1-preview.

The R1 distilled model, released around the same time, is stronger than s1, but uses 800 times as much data. On AIME24, s1 also comes close to its teacher, Gemini 2.0 Flash Thinking, which suggests the distillation largely worked.

How the data is picked makes a big difference

DataAIME24MATH500GPQA
Random 1,000 (quality only)36.790.652.0
Most diverse 1,000 (diversity only)26.791.254.6
Longest-reasoning 1,000 (difficulty only)33.390.459.6
All 59,00053.392.858.1
s1K50.093.057.6

Compiled from Table 2 of the paper; every score in this table caps thinking at about 30,000 tokens to keep the model from getting stuck in a loop. In %.

Picking questions by any single principle loses 13 to 23 percentage points on AIME24. s1K, which uses all three principles together, does about as well as the model trained on all 59,000 questions, and the latter takes 394 GPU hours to train while s1K takes only 7.

How to control thinking length

MethodControlScalingPerformance
Budget forcing100%1556.7
Token count in the prompt40%−2440.0
Token count in the prompt + budget forcing100%1340.0
Step count in the prompt60%336.7
Step count in the prompt + budget forcing100%636.7
Short or long in the prompt50%2536.7
Rejection sampling100%−3540.0

Compiled from Table 3 of the paper; AIME24.

Budget forcing does well on all three metrics, with 100% control, positive scaling and the highest accuracy. The other methods each have problems:

  • Specifying a token count: the model can't keep accurate count of how many tokens it has written, even after specific training, and without budget forcing it can't be kept in check.
  • Specifying a step count: the model is very good at gaming it. Ask it for fewer steps and it makes each step longer; ask for more steps and it makes each step shorter, so the total token count stays about the same.
  • Specifying short or long: telling the model to think longer does make it think more, and its scaling is even the best, but the accuracy isn't high.
  • Rejection sampling: the biggest surprise. The looser the length limit, the lower the accuracy, with a scaling of −35. On inspection, the authors found that short generations were mostly cases where the model headed in the right direction from the start; long generations were often cases where the model went wrong and kept backtracking and doubting itself. Picking only answers of a certain length amounts to picking cases of different difficulty.

As for the word appended afterward, the authors compared several:

Handling at end of thinkingAIME24MATH500GPQA
No extension50.093.057.6
Swallowed 2 times, nothing appended50.090.255.1
Swallowed 2 times, "Alternatively" appended50.092.259.6
Swallowed 2 times, "Hmm" appended50.093.059.6
Swallowed 2 times, "Wait" appended53.393.059.6

Compiled from Table 4 of the paper; in %.

Just removing the end-of-thinking token and appending nothing actually makes things worse. "Wait" is the best overall, and the only one that improves AIME24.

Why 1,000 examples are enough

The authors' hypothesis is that the ability to reason is already in the model. Pretraining used trillions of tokens, which already contain a great deal of reasoning data; the 1,000-example fine-tuning merely brings that ability out, and budget forcing then amplifies it at inference time. This is similar to the "Superficial Alignment Hypothesis" put forward by LIMA, which found that 1,000 examples were enough for a model to learn to respond the way users prefer.

Limitations

  • It levels off: past a certain point, extending with budget forcing stops helping, and doing it too many times sends the model into repetitive loops. The improvements the authors suggest include rotating through different words, and adding a repetition penalty or raising the temperature.
  • Context length: however long the thinking, it can't exceed the model's context window. When the model was asked to think for 512 steps on AIME24, 12 of the 30 questions produced output longer than the context length, and the score dropped sharply as a result.
  • Parallel scaling can fill the gap: the authors tried REBASE, a tree search guided by a process reward model, combined with majority vote; it scaled better than plain majority vote and, in this comparison, also beat sequential scaling, though every step requires an extra pass of the reward model.

Also, every reasoning trace in s1K was written by Gemini, so the model can roughly only learn up to its teacher's level; this shows up more clearly with s1.1 below.

What came next

Seven days after s1 came out, the authors released s1.1. Same 1,000 questions, but with reasoning traces regenerated by DeepSeek-R1, and the same training method:

ModelMATH500GPQAAIME24AIME25
s1 (no budget forcing)92.656.650.026.7
s1 ("Wait" appended 4 times)92.258.656.736.7
s1.1 (no budget forcing)94.460.656.750.0
s1.1 ("Wait" appended 2 times)95.463.656.750.0

Compiled from Table 5 in the paper's appendix, selected rows; in %.

Changing only the teacher, s1.1 with no budget forcing at all matches s1's best scores with budget forcing; on AIME25, the new 2025 problems, it jumps from 26.7% to 50.0%. The authors also tried Claude 3.7 as the teacher, with worse results than R1. Interestingly, once s1.1 gets "Wait" appended, its AIME scores stop improving.

The same table also lists a few new rivals. LIMO, fine-tuned on 817 examples, scores 56.3% on AIME24, reaching a conclusion similar to s1's. OpenAI's o3-mini simply turns reasoning effort into three settings, with low, medium and high scoring 56.3%, 75.8% and 83.8% on AIME24; controlling how long the model thinks became a product option.

Conclusion

s1 takes 1,000 carefully chosen questions, fine-tunes Qwen2.5-32B-Instruct on them for 26 minutes, and then controls thinking length with budget forcing, openly reproducing o1's "think longer, answer better" curve. AIME24 goes from 50% to 56.7%, beating o1-preview on competition math.

The most instructive part is the data ablation table. Pick 1,000 questions by any one criterion alone, whether random, most diverse or longest, and the result is much worse than using all three together; training on all 59,000 questions gains only a little more. Having little data isn't the problem; picking it well is what matters.

As for that "Wait," it does help, but it can only stretch things so far, and switching to a better teacher makes a bigger difference. What really lets s1 keep thinking is less that one word than the reasoning habits that Gemini, and later R1, wrote into those 1,000 questions.