s1: Simple test-time scaling
- Publication
- EMNLP 2025Conference
- First submitted to arXiv
- arXiv:2501.19393
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{modeltunings12025,
title = {{s1: Simple test-time scaling}},
author = {Niklas Muennighoff and Zitong Yang and Weijia Shi and Xiang Lisa Li and Li Fei-Fei and Hannaneh Hajishirzi and Luke Zettlemoyer and Percy Liang and Emmanuel Candès and Tatsunori Hashimoto},
year = {2025},
eprint = {2501.19393},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2501.19393}
}
In September 2024, OpenAI released o1 and showed off a lovely curve: the longer the model thinks, the more accurate its answers. But OpenAI didn't disclose how it was done, saying only that it used large-scale reinforcement learning.
Over the following months, many teams tried to reproduce it. Some used Monte Carlo tree search, some had multiple agents debate each other; DeepSeek-R1, in January 2025, used millions of samples and several stages of reinforcement learning to catch up with o1's scores. Yet among all these reproductions, no one had openly reproduced that "think longer, answer better" curve.
Muennighoff, Yang and colleagues from Stanford, the University of Washington, AI2 and elsewhere asked a very direct question: what is the simplest way to get both that curve and strong reasoning ability? Their answer is surprisingly simple. 1,000 examples, 26 minutes of fine-tuning on 16 H100s, plus one "Wait."

Defining the problem
The authors divide methods that spend more compute at inference time into two kinds:
- Sequential: later computation builds on earlier results, as in writing out a very long chain of reasoning.
- Parallel: computations run independently and are aggregated at the end, as in majority vote.
The authors focus on sequential methods, on the intuition that later computation can make use of earlier intermediate results, reason more deeply, and revise again and again.
To compare methods, accuracy alone isn't enough; you also need to know whether the compute can be controlled, and whether accuracy rises as compute increases. The authors define three metrics. For a given method, run it several times on the same benchmark with different amounts of compute , giving accuracy :
Control is the fraction of runs in which the thinking tokens actually used fall within the specified range; Scaling is the average slope of the curve, which has to be positive to be useful; Performance is the highest accuracy reached.
Solving the problem
s1K: picking 1,000 questions

First, collect 59,000 questions. There are 16 sources; the largest is NuminaMATH, with 30,660 math problems, along with past AIME problems (1983 to 2021), OlympicArena's interdisciplinary olympiad problems (4,250), OmniMath's competition math (4,238), and AGIEval's standardized exams such as the SAT and LSAT (2,385). The authors also made two sets of their own: 182 probability questions from Stanford's statistics PhD qualifying exam, and 23 hard brain teasers of the kind common in quantitative trading interviews.
Every question was run through Google's Gemini 2.0 Flash Thinking to generate a reasoning trace and a solution, giving 59,029 "question, reasoning, solution" triples. Any data sharing a run of 8 consecutive words with the benchmark questions (MATH500, GPQA Diamond, AIME24) was removed, and duplicate questions were dropped as well.
Then three principles are used to filter:
- Quality: remove those with API errors, leaving 54,116 questions; then remove those with formatting problems, such as ASCII art, references to nonexistent images, or garbled question numbering, leaving 51,581. Of these, 384 come from high-quality datasets that need no further filtering and go straight into the final list.
- Difficulty: have Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct each attempt every question once, with Claude 3.5 Sonnet grading them against the reference answer. If either model gets it right, the question is treated as too easy and dropped, leaving 24,496. The length of the reasoning trace is also used as a measure of difficulty, on the assumption that harder questions need longer thinking.
- Diversity: have Claude 3.5 Sonnet assign each question to a domain according to the American Mathematical Society's Mathematics Subject Classification. Each time, first pick a domain at random, then pick a question from that domain, with longer reasoning traces more likely to be picked; repeat until there are 1,000 questions, which end up covering 50 domains.
The grading model judges only 53.6% of the reasoning traces in s1K to be correct. The authors deliberately keep these mistakes; what they want is the reasoning process, not perfectly correct solutions.
Budget forcing

The method for controlling thinking length is called budget forcing, and it intervenes directly during decoding:
- Setting a maximum: once the number of thinking tokens exceeds the limit, append the end-of-thinking token directly, adding "Final Answer:" if needed, to force the model to hand over its best answer so far.
- Setting a minimum: when the model tries to stop thinking, swallow the end-of-thinking token and append "Wait" instead, so it keeps thinking.
The figure above is one example. The model counted once, concluded that raspberry has only two r's, and tried to stop; after "Wait" was appended, it reread the question and found the third r.
What the model writes after "Wait" depends on what it learned from its training data. The reasoning traces in s1K are full of sentences like "wait a moment" and "let me check that again," so when the model sees "Wait," it goes on to check its earlier reasoning.
The authors also compare several other ways of controlling length:
- Token count in the prompt: tell the model the maximum number of tokens to think for.
- Step count in the prompt: tell the model the maximum number of steps to think for, with each step about 100 tokens.
- Short or long in the prompt: two prompts, one telling the model to think briefly and one telling it to think longer.
- Rejection sampling: keep sampling until one generation happens to fall within the budget.
Training
The base model is Qwen2.5-32B-Instruct. The thinking part starts with <|im_start|>think and the answer part with <|im_start|>answer. Training runs on s1K for 5 epochs with batch size 16, 315 steps in total, at learning rate , with warmup over the first 5% followed by cosine decay to 0; the loss is computed only on the reasoning trace and the solution, not on the question.
Training took 26 minutes on 16 H100s, about 7 GPU hours.
Discussion
Think longer, answer better

Stretching the thinking with budget forcing lifts AIME24 from 50% with no intervention to 56.7%. But by the sixth "Wait," the curve has gone flat; swallow the end-of-thinking token too often and the model falls into repetitive loops instead of continuing to reason.
The right panel compares the two ways of scaling. The base model, Qwen2.5-32B-Instruct, is sampled 64 times per question at temperature 1, with majority votes over 2 to 64 answers, and its accuracy never catches up with s1's sequential scaling. The authors take this as confirming their initial intuition that sequential scaling is more effective than parallel scaling.
Results
| Model | Reasoning examples | AIME24 | MATH500 | GPQA |
|---|---|---|---|---|
| o1-preview | Undisclosed | 44.6 | 85.5 | 73.3 |
| o1 | Undisclosed | 74.4 | 94.8 | 77.3 |
| Gemini 2.0 Flash Thinking | Undisclosed | 60.0 | - | - |
| Qwen2.5-32B-Instruct | - | 26.7 | 84.0 | 49.0 |
| DeepSeek-R1 | Far more than 800K | 79.8 | 97.3 | 71.5 |
| R1 distilled (32B) | 800K | 72.6 | 94.3 | 62.1 |
| Bespoke-32B | 17K | 63.3 | 93.0 | 58.1 |
| s1 (no budget forcing) | 1,000 | 50.0 | 92.6 | 56.6 |
| s1-32B | 1,000 | 56.7 | 93.0 | 59.6 |
Compiled from Table 1 of the paper, selected rows; in %.
With just 1,000 extra training examples, s1-32B scores 30 percentage points higher than the base model on AIME24. On both AIME24 and MATH500 it beats o1-preview; the abstract's claim of being "up to 27%" higher is the relative gap of 56.7 versus 44.6 on AIME24. On GPQA, though, it's still a dozen or so percentage points below o1-preview.
The R1 distilled model, released around the same time, is stronger than s1, but uses 800 times as much data. On AIME24, s1 also comes close to its teacher, Gemini 2.0 Flash Thinking, which suggests the distillation largely worked.
How the data is picked makes a big difference
| Data | AIME24 | MATH500 | GPQA |
|---|---|---|---|
| Random 1,000 (quality only) | 36.7 | 90.6 | 52.0 |
| Most diverse 1,000 (diversity only) | 26.7 | 91.2 | 54.6 |
| Longest-reasoning 1,000 (difficulty only) | 33.3 | 90.4 | 59.6 |
| All 59,000 | 53.3 | 92.8 | 58.1 |
| s1K | 50.0 | 93.0 | 57.6 |
Compiled from Table 2 of the paper; every score in this table caps thinking at about 30,000 tokens to keep the model from getting stuck in a loop. In %.
Picking questions by any single principle loses 13 to 23 percentage points on AIME24. s1K, which uses all three principles together, does about as well as the model trained on all 59,000 questions, and the latter takes 394 GPU hours to train while s1K takes only 7.
How to control thinking length
| Method | Control | Scaling | Performance |
|---|---|---|---|
| Budget forcing | 100% | 15 | 56.7 |
| Token count in the prompt | 40% | −24 | 40.0 |
| Token count in the prompt + budget forcing | 100% | 13 | 40.0 |
| Step count in the prompt | 60% | 3 | 36.7 |
| Step count in the prompt + budget forcing | 100% | 6 | 36.7 |
| Short or long in the prompt | 50% | 25 | 36.7 |
| Rejection sampling | 100% | −35 | 40.0 |
Compiled from Table 3 of the paper; AIME24.
Budget forcing does well on all three metrics, with 100% control, positive scaling and the highest accuracy. The other methods each have problems:
- Specifying a token count: the model can't keep accurate count of how many tokens it has written, even after specific training, and without budget forcing it can't be kept in check.
- Specifying a step count: the model is very good at gaming it. Ask it for fewer steps and it makes each step longer; ask for more steps and it makes each step shorter, so the total token count stays about the same.
- Specifying short or long: telling the model to think longer does make it think more, and its scaling is even the best, but the accuracy isn't high.
- Rejection sampling: the biggest surprise. The looser the length limit, the lower the accuracy, with a scaling of −35. On inspection, the authors found that short generations were mostly cases where the model headed in the right direction from the start; long generations were often cases where the model went wrong and kept backtracking and doubting itself. Picking only answers of a certain length amounts to picking cases of different difficulty.
As for the word appended afterward, the authors compared several:
| Handling at end of thinking | AIME24 | MATH500 | GPQA |
|---|---|---|---|
| No extension | 50.0 | 93.0 | 57.6 |
| Swallowed 2 times, nothing appended | 50.0 | 90.2 | 55.1 |
| Swallowed 2 times, "Alternatively" appended | 50.0 | 92.2 | 59.6 |
| Swallowed 2 times, "Hmm" appended | 50.0 | 93.0 | 59.6 |
| Swallowed 2 times, "Wait" appended | 53.3 | 93.0 | 59.6 |
Compiled from Table 4 of the paper; in %.
Just removing the end-of-thinking token and appending nothing actually makes things worse. "Wait" is the best overall, and the only one that improves AIME24.
Why 1,000 examples are enough
The authors' hypothesis is that the ability to reason is already in the model. Pretraining used trillions of tokens, which already contain a great deal of reasoning data; the 1,000-example fine-tuning merely brings that ability out, and budget forcing then amplifies it at inference time. This is similar to the "Superficial Alignment Hypothesis" put forward by LIMA, which found that 1,000 examples were enough for a model to learn to respond the way users prefer.
Limitations
- It levels off: past a certain point, extending with budget forcing stops helping, and doing it too many times sends the model into repetitive loops. The improvements the authors suggest include rotating through different words, and adding a repetition penalty or raising the temperature.
- Context length: however long the thinking, it can't exceed the model's context window. When the model was asked to think for 512 steps on AIME24, 12 of the 30 questions produced output longer than the context length, and the score dropped sharply as a result.
- Parallel scaling can fill the gap: the authors tried REBASE, a tree search guided by a process reward model, combined with majority vote; it scaled better than plain majority vote and, in this comparison, also beat sequential scaling, though every step requires an extra pass of the reward model.
Also, every reasoning trace in s1K was written by Gemini, so the model can roughly only learn up to its teacher's level; this shows up more clearly with s1.1 below.
What came next
Seven days after s1 came out, the authors released s1.1. Same 1,000 questions, but with reasoning traces regenerated by DeepSeek-R1, and the same training method:
| Model | MATH500 | GPQA | AIME24 | AIME25 |
|---|---|---|---|---|
| s1 (no budget forcing) | 92.6 | 56.6 | 50.0 | 26.7 |
| s1 ("Wait" appended 4 times) | 92.2 | 58.6 | 56.7 | 36.7 |
| s1.1 (no budget forcing) | 94.4 | 60.6 | 56.7 | 50.0 |
| s1.1 ("Wait" appended 2 times) | 95.4 | 63.6 | 56.7 | 50.0 |
Compiled from Table 5 in the paper's appendix, selected rows; in %.
Changing only the teacher, s1.1 with no budget forcing at all matches s1's best scores with budget forcing; on AIME25, the new 2025 problems, it jumps from 26.7% to 50.0%. The authors also tried Claude 3.7 as the teacher, with worse results than R1. Interestingly, once s1.1 gets "Wait" appended, its AIME scores stop improving.
The same table also lists a few new rivals. LIMO, fine-tuned on 817 examples, scores 56.3% on AIME24, reaching a conclusion similar to s1's. OpenAI's o3-mini simply turns reasoning effort into three settings, with low, medium and high scoring 56.3%, 75.8% and 83.8% on AIME24; controlling how long the model thinks became a product option.
Conclusion
s1 takes 1,000 carefully chosen questions, fine-tunes Qwen2.5-32B-Instruct on them for 26 minutes, and then controls thinking length with budget forcing, openly reproducing o1's "think longer, answer better" curve. AIME24 goes from 50% to 56.7%, beating o1-preview on competition math.
The most instructive part is the data ablation table. Pick 1,000 questions by any one criterion alone, whether random, most diverse or longest, and the result is much worse than using all three together; training on all 59,000 questions gains only a little more. Having little data isn't the problem; picking it well is what matters.
As for that "Wait," it does help, but it can only stretch things so far, and switching to a better teacher makes a bigger difference. What really lets s1 keep thinking is less that one word than the reasoning habits that Gemini, and later R1, wrote into those 1,000 questions.