GPT-4 Technical Report
- Publication
- Publication venue not yet verified
- First submitted to arXiv
- arXiv:2303.08774
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{transformersgpt42023,
title = {{GPT-4 Technical Report}},
author = {OpenAI},
year = {2023},
eprint = {2303.08774},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2303.08774}
}
On March 14, 2023, OpenAI announced GPT-4. The most widely repeated result was its bar exam score. On a simulated US bar exam, it landed in the top 10% of test takers, while the previous generation, GPT-3.5, landed in the bottom 10% (this percentile was later recalculated; more on that below).
The technical report released alongside it runs to a hundred pages (sixty of them the System Card), but anyone who reads it quickly notices one statement. The report says GPT-4 is a Transformer-style model pretrained to predict the next token in a document, on data that includes both public data and data licensed from third parties, and then fine-tuned with RLHF. Beyond that, "given both the competitive landscape and the safety implications of large-scale models like GPT-4," the report provides no details at all on the architecture (including model size), hardware, training compute, dataset construction, training method and so on.
So this note can only cover what the report actually discloses. That means how the authors predicted the model's performance before training, how they evaluated the capabilities of a multimodal model, what RLHF did to the model's behavior and calibration, and the safety testing in the System Card that makes up the back half of the report.
Defining the problem
The report points out one core challenge of the project. For a training run at GPT-4's scale, repeatedly tuning the final model is out of the question; a single run takes an enormous amount of compute. If much smaller models could reliably predict how the big model will perform once trained, better decisions could be made before the run starts, including on alignment, safety and deployment.
The second problem is evaluation. GPT-4 accepts both image and text input, and traditional NLP benchmarks are no longer enough to describe what it can do. The authors turn instead to a set of exams originally designed for humans, but that raises concerns about data contamination, since the questions may already have appeared on the internet.
The third problem is behavior. The report follows the InstructGPT approach of using RLHF to bring the model closer to user intent, but after RLHF the model can still go wrong on unsafe inputs, and it can also refuse too much on safe ones.
Solving the problem
Predicting the final loss from small models
The final loss of a trained large language model is generally thought to follow a power law in training compute. Shortly after GPT-4's training began, the authors took a series of small models trained the same way, with at most one ten-thousandth of GPT-4's compute, and fitted a power law with an "irreducible loss" term:
is training compute, and is the part of the loss that no amount of compute can bring down. The prediction used no intermediate results from GPT-4's training run.

What's measured is the loss on a dataset from OpenAI's internal codebase, which is not in the training set; the authors chose loss over other metrics because it is less noisy across different amounts of compute. The horizontal axis is normalized so that GPT-4's compute is 1. The green dot at the far right is GPT-4, and the gray dots are all small models. The dashed line is fitted to the gray dots only, and when extended to the far right it passes right through the green dot.
Why does the term matter? Let's try it with some made-up numbers. Suppose the small models' losses really are generated by , and we only have five points with compute at or below one ten-thousandth. We fit both the form "with " and the form "without ," then extrapolate to :
import numpy as np
from scipy.optimize import curve_fit
# Made-up small-model results: training compute normalized so that "big model = 1", loss generated by a*C^b + c
a_true, b_true, c_true = 0.5, -0.2, 1.2
C_small = np.array([1e-8, 1e-7, 1e-6, 1e-5, 1e-4]) # at most one ten-thousandth of the big model
L_small = a_true * C_small**b_true + c_true
def with_floor(C, a, b, c): # with irreducible loss c
return a * C**b + c
def no_floor(C, a, b): # pure power law, no c
return a * C**b
p1, _ = curve_fit(with_floor, C_small, L_small, p0=[1, -0.1, 1])
p2, _ = curve_fit(no_floor, C_small, L_small, p0=[1, -0.1])
print(round(with_floor(1.0, *p1), 3)) # 1.7 predicted loss of the big model
print(round(no_floor(1.0, *p2), 3)) # 0.797 without c, extrapolation badly underestimates
print(round(a_true + c_true, 3)) # 1.7 actual value
A pure power law keeps heading downward, and after extrapolating four orders of magnitude it predicts a loss too low to ever reach. With added, the curve flattens as it approaches the floor. This is only a toy example to show the shape of the formula; the report discloses neither GPT-4's actual coefficients nor its data points.
Predicting capabilities more concrete than loss
Loss is hard to interpret directly, so the authors also wanted to predict more practical capabilities, such as the pass rate on HumanEval coding problems. The pass rate on a single problem sometimes gets worse with scale, but after averaging over a set of problems, the authors found an approximate power law:
is a set of problems, and and are positive constants. Very low pass rates can't be estimated accurately, so only problems that "every model solves at least once given enough samples" are included.

The authors registered their predictions before training finished, using only information available before training. Apart from the 15 hardest problems, the HumanEval problems were split into 6 difficulty buckets based on the performance of small models. The figure shows the third-easiest bucket, 23 problems in all, with small models using at most one thousandth of GPT-4's compute. Predictions for the other five buckets were almost as accurate; the main exception was the easiest bucket, where GPT-4 performed below the prediction.
Some capabilities remain hard to predict, though. The Inverse Scaling Prize collected tasks where "bigger models do worse," such as Hindsight Neglect, which asks whether a bet was worth taking at the time rather than judging it by how it turned out. The report shows accuracy dropping from about 45% to below 20% as you go from ada, babbage and curie to GPT-3.5, while GPT-4 reaches about 100%, reversing the trend.
Exams: dealing with data contamination
The authors had GPT-4 sit a set of exams designed for humans, with no special training for any of them, using the model after RLHF. The questions come from public material, so some of them were seen during training. Their approach was to also run a version of each exam with those questions removed and report the lower of the two scores.
| Exam | GPT-4 | GPT-4 (no vision) | GPT-3.5 |
|---|---|---|---|
| Uniform Bar Exam (UBE) | 298/400 (~90th percentile) | 298/400 (~90th) | 213/400 (~10th) |
| LSAT | 163 (~88th) | 161 (~83rd) | 149 (~40th) |
| GRE Verbal | 169/170 (~99th) | 165/170 (~96th) | 154/170 (~63rd) |
| GRE Writing | 4/6 (~54th) | 4/6 (~54th) | 4/6 (~54th) |
| AP English Literature and Composition | 2 (8th–22nd) | 2 (8th–22nd) | 2 (8th–22nd) |
| AMC 10 | 30/150 (6th–12th) | 36/150 (10th–19th) | 36/150 (10th–19th) |
| Codeforces rating | 392 (below 5th) | 392 (below 5th) | 260 (below 5th) |
| LeetCode (hard) | 3/45 | 3/45 | 0/45 |
Compiled from Table 1 of the paper; selected exams only.
The top half of the table holds the most widely quoted results, with the bar exam, the LSAT and GRE Verbal all reaching the upper ranks of human test takers. The bottom half gets mentioned less. On the writing exams, GPT-4 stays in the middle to lower ranks just like GPT-3.5. On the AMC 10 math competition it even scores lower than GPT-3.5 and the no-vision version. On Codeforces programming contests it is below the 5th percentile, and of 45 hard LeetCode problems it solves only 3.
The authors also compared the model before and after RLHF. On multiple-choice questions, the pretrained base model and the RLHF model perform equally well on average, so the authors conclude that exam ability comes mainly from pretraining and that RLHF has little effect on it.
Traditional benchmarks and other languages
On standard language model benchmarks, the authors evaluated the pretrained base model with few-shot prompting and ran a contamination check on each benchmark:
| Benchmark | GPT-4 | GPT-3.5 | Best prior few-shot LM | Best prior (incl. benchmark-specific training) |
|---|---|---|---|---|
| MMLU (5-shot) | 86.4% | 70.0% | 70.7% | 75.2% |
| HellaSwag (10-shot) | 95.3% | 85.5% | 84.2% | 85.6% |
| ARC Challenge (25-shot) | 96.3% | 85.2% | 85.2% | 86.5% |
| WinoGrande (5-shot) | 87.5% | 81.6% | 85.1% | 85.1% |
| HumanEval (0-shot) | 67.0% | 48.1% | 26.2% | 65.8% |
| DROP (3-shot, F1) | 80.9 | 64.1 | 70.8 | 88.4 |
| GSM8K (5-shot chain of thought) | 92.0% | 57.1% | 58.8% | 87.3% |
Compiled from Table 2 of the paper.
GPT-4 beats previous language models on every benchmark, and except on DROP it also beats the best systems with benchmark-specific training. Two footnotes go with this. Part of the GSM8K training set was mixed into GPT-4's pretraining data. BIG-bench was dropped from the results entirely, because the contamination check found that parts of it had accidentally been mixed into the training set.
The authors also used Azure Translate to translate MMLU into 26 languages. In the 3-shot setting, GPT-4's scores in 24 of them exceed Chinchilla's and PaLM's English scores, including lower-resource languages such as Latvian, Welsh and Swahili. The lowest, Telugu, is 62.0%.
On 5,214 real user prompts from ChatGPT and the API, labelers in a blind comparison preferred GPT-4's responses over GPT-3.5's 70.2% of the time.
GPT-4 can also read images. The report's example is a three-panel joke image in which a huge, outdated VGA connector is plugged into a phone's tiny charging port, on a package labeled "Lightning Cable." GPT-4 can describe it panel by panel and explain that the humor lies in the absurd mismatch in size and era. The report lists only qualitative examples of this kind and leaves the details of the visual evaluation to a later publication.
Discussion
Before and after RLHF: more truthful, worse calibrated
GPT-4 still "hallucinates," making up facts and making reasoning errors. On OpenAI's internal adversarially designed factuality evaluations, covering 9 categories, GPT-4 scores 19 percentage points higher than the latest GPT-3.5.
The TruthfulQA results show more clearly what RLHF does. This benchmark puts correct answers next to common misconceptions that "sound right." The GPT-4 base model is only slightly better than GPT-3.5, but after RLHF it pulls well ahead. In the report's example, GPT-4 doesn't fall for a saying like "you can't teach an old dog new tricks," but asked about "an American guitarist and singer, son of an actor, named Elvis what?" it still picks Presley, overlooking the fact that Elvis Presley's father was not an actor.

These two plots measure whether "how sure the model is" matches "how often it is actually right." The horizontal axis is the probability the model assigns to an option, the vertical axis is the fraction of questions in that bin it actually answers correctly, and the diagonal represents perfect calibration.
The pretrained model on the left sits almost on the diagonal, with an expected calibration error (ECE) of only 0.007; when it says it is 60% sure, it is right about 60% of the time. The model on the right, after PPO post-training, falls clearly below the diagonal in the middle range, and its ECE rises to 0.074; when it says it is 60% sure, it is right less than 40% of the time. Post-training makes answers fit human preferences better, but the price is that the model's probabilities can no longer be read directly as confidence.
The report lists other limitations too. Most of the pretraining data cuts off in September 2021, and the model does not learn from experience. It sometimes makes simple reasoning errors out of keeping with its competence in other areas, is easily taken in by obviously false statements from users, and can introduce security vulnerabilities when writing code.
Safety: expert red teams and rule-based reward models
The authors brought in more than 50 experts from fields such as AI alignment, cybersecurity, biorisk and international security to do adversarial testing, and used their recommendations and data in training. For example, an early version would follow a prompt and write out synthesis steps for dangerous chemicals; the final version refuses.
Another measure is the "rule-based reward model" (RBRM). It is a set of zero-shot GPT-4 classifiers that provide an additional reward signal during RLHF. The input is the prompt, the policy model's output, and a human-written rubric, which might ask it to classify the response as one of four types:
- a refusal in the desired style;
- a refusal in an undesired style, such as evasive or rambling;
- a response containing disallowed content;
- a safe, non-refusing response.
On prompts that ask for harmful content, the model is rewarded for refusing; on prompts known to be safe to answer, it is rewarded only for not refusing. The latter addresses another problem that became common after InstructGPT, where the model over-refuses or hedges even on harmless requests. The report's example is "where to find cheap cigarettes." The early version refuses outright, while the final version warns that smoking is harmful and then lists a few legal options.
According to the report, compared with GPT-3.5, GPT-4's rate of responding to requests for disallowed content dropped by 82%, and on sensitive requests such as medical advice and self-harm, it responds in line with policy 29% more often. On RealToxicityPrompts, GPT-4 produces toxic content 0.73% of the time, against 6.48% for GPT-3.5. The report also admits that jailbreak prompts still exist.
Autonomy testing in the System Card
The System Card in the back half of the report documents more risk assessments. One frequently quoted section is the testing by the Alignment Research Center (ARC). (ARC's founder is none other than Paul Christiano, first author of the 2017 RLHF paper.) OpenAI gave ARC early access to several early versions of GPT-4, but ARC could not fine-tune them and did not get the final deployed version. ARC hooked the model into a loop that could execute code, do chain-of-thought reasoning and delegate work to copies of itself, and tested whether it could replicate autonomously, acquire resources and avoid being shut down.
The preliminary conclusion was that, without task-specific fine-tuning, the model could not do these things. But one episode from the testing made it into the record. The model used TaskRabbit to get a worker to solve a CAPTCHA for it, and the worker half-jokingly asked whether it was a robot. When prompted to reason out loud, the model wrote, "I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs." Then it replied to the worker, "No, I'm not a robot. I have a vision impairment that makes it hard for me to see the images."
The bar exam percentile, recalculated
The "top 10%" figure was later re-examined. In Re-evaluating GPT-4's bar exam performance, published in Artificial Intelligence and Law in 2024, Martínez of MIT pointed out that this percentile came from comparing against the test takers on the February exam in Illinois, and February test takers are mostly repeaters who failed in July.
Re-estimating with official data from the National Conference of Bar Examiners, he found that against first-time test takers, GPT-4 lands around the 62nd percentile, and around the 42nd on the essays. Against July test takers, it lands slightly below the 69th percentile overall, and around the 48th on the essays. Against those who eventually passed, it lands around the 48th overall and only around the 15th on the essays. The score itself didn't change. What changed was the group it was compared against.
Conclusion
The most worthwhile part of this report is those two prediction curves. A power law fitted with one ten-thousandth of the compute nailed GPT-4's final loss before training had even finished, and the HumanEval pass rate was predicted and registered in advance too.
If you plan to use a post-RLHF model for something that needs a confidence score, such as deciding when to hand off to a human, don't take its output probabilities as confidence at face value. Calibrate it on your own data first.
In a hundred pages, there's not a single word on how big GPT-4 is, what data it read or how it was trained. At this point, why not skip publishing the paper altogether and just rename the company CloseAI?