{"notes":[{"id":"deepseek/deepseek-ocr-2","category":"deepseek","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"DeepSeek-OCR 2","subtitle":"Where the Eye Goes Next","author":"Z. Yuan","paperDate":"2026-01-28","publishedOn":"2026-11-08","world":"external/deepseek-ocr2","paperTitle":"DeepSeek-OCR 2: Visual Causal Flow","paperUrl":"https://arxiv.org/abs/2601.20552","bibliography":{"title":"DeepSeek-OCR 2: Visual Causal Flow","authors":["Haoran Wei","Yaofeng Sun","Yukun Li"],"url":"https://arxiv.org/abs/2601.20552","pdfUrl":"https://arxiv.org/pdf/2601.20552v1","codeUrl":"https://github.com/deepseek-ai/DeepSeek-OCR-2","preprint":{"id":"2601.20552","submittedOn":"2026-01-28","url":"https://arxiv.org/abs/2601.20552"},"publication":null,"citation":"@misc{deepseekdeepseekocr22026,\n  title = {{DeepSeek-OCR 2: Visual Causal Flow}},\n  author = {Haoran Wei and Yaofeng Sun and Yukun Li},\n  year = {2026},\n  eprint = {2601.20552},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2601.20552}\n}\n","citationPath":"/content/papers/citations/deepseek/deepseek-ocr-2.02901bc9ba9f.bib"},"thumbnail":{"src":"/content/papers/assets/2ba25df8f4e3579425551e93bdce7b8b0f180bb349cef46511dce7f44ffd68f3.png","width":1194,"height":483},"locale":"en","path":"/content/papers/en/deepseek/deepseek-ocr-2.e97fc23e2ca9.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"swapping-clip-for-a-language-model","text":"Swapping CLIP for a Language Model","depth":3},{"id":"the-mask-decides-who-sees-whom","text":"The Mask Decides Who Sees Whom","depth":3},{"id":"how-many-tokens-does-a-page-take","text":"How Many Tokens Does a Page Take?","depth":3},{"id":"training","text":"Training","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"omnidocbench-v15","text":"OmniDocBench v1.5","depth":3},{"id":"the-nine-document-types-one-by-one","text":"The Nine Document Types, One by One","depth":3},{"id":"what-to-watch-in-production","text":"What to Watch in Production","depth":3},{"id":"the-authors-vision-and-later-results","text":"The Authors' Vision and Later Results","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"deepseek/deepseek-ocr","relation":"referenced","topic":""}]},{"id":"deepseek/deepseek-ocr","category":"deepseek","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"DeepSeek-OCR","subtitle":"A Picture Is Worth a Thousand Words","author":"Z. Yuan","paperDate":"2025-10-21","publishedOn":"2026-03-09","world":"external/deepseek-ocr","paperTitle":"DeepSeek-OCR: Contexts Optical Compression","paperUrl":"https://arxiv.org/abs/2510.18234","bibliography":{"title":"DeepSeek-OCR: Contexts Optical Compression","authors":["Haoran Wei","Yaofeng Sun","Yukun Li"],"url":"https://arxiv.org/abs/2510.18234","pdfUrl":"https://arxiv.org/pdf/2510.18234v1","codeUrl":"https://github.com/deepseek-ai/DeepSeek-OCR","preprint":{"id":"2510.18234","submittedOn":"2025-10-21","url":"https://arxiv.org/abs/2510.18234"},"publication":null,"citation":"@misc{deepseekdeepseekocr2025,\n  title = {{DeepSeek-OCR: Contexts Optical Compression}},\n  author = {Haoran Wei and Yaofeng Sun and Yukun Li},\n  year = {2025},\n  eprint = {2510.18234},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2510.18234}\n}\n","citationPath":"/content/papers/citations/deepseek/deepseek-ocr.66e8e5a7db8d.bib"},"thumbnail":{"src":"/content/papers/assets/6f53ecfe63f39804454a69ef4468c93e8018fa1afbe08d1bb52b7e8d52d194c3.png","width":1288,"height":350},"locale":"en","path":"/content/papers/en/deepseek/deepseek-ocr.37f4dfc7859e.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"overall-architecture","text":"Overall Architecture","depth":3},{"id":"walking-one-10241024-page-through","text":"Walking One 1024×1024 Page Through","depth":3},{"id":"the-decoder-recovering-text-from-a-compressed-representation","text":"The Decoder: Recovering Text from a Compressed Representation","depth":3},{"id":"data-and-training","text":"Data and Training","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"compression-experiments-100-pages-from-fox","text":"Compression Experiments: 100 Pages from Fox","depth":3},{"id":"real-documents-omnidocbench","text":"Real Documents: OmniDocBench","depth":3},{"id":"optical-compression-and-forgetting","text":"Optical Compression and Forgetting","depth":3},{"id":"discussion-after-release","text":"Discussion After Release","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"model-evaluation/linguistic-crutch","relation":"referenced","topic":""}]}],"text":"How does a person read a three-column newspaper?\n\nWe don't sweep from the top-left corner straight down to the bottom-right. We read the headline first, then follow the first column down; at the bottom we jump back up to the top of the second column; when we see “continued on page 5,” our eyes jump there. Where we look next depends on what we just read.\n\nVLMs don't read that way. The image is cut into a grid of patches, laid out in one row in a fixed “left to right, top to bottom” order, given positional encodings, and handed to the language model. For ordinary photos that works fine; for documents with complex layouts, this order often differs from the logical order of the content.\n\nDeepSeek-OCR 2 wants the encoder to decide for itself **where to look first and where to look next**. It is a technical report that DeepSeek-AI uploaded on January 28, 2026; this note follows v1. The code and weights were open-sourced at the same time.\n\n\n> [!TIP]\n> \n> For the previous generation's DeepEncoder and its optical compression design, start here:\n> \n> - **[[25.10] DeepSeek-OCR: A Picture Is Worth a Thousand Words](/blog/papers/deepseek/deepseek-ocr)**\n\n\n## Defining the Problem\n\nThe authors start from an observation: human eye movements are causal, with each next fixation depending on what came before; yet today's VLM encoders flatten a 2D image into a 1D sequence in a fixed order that has nothing to do with the content.\n\nLanguage models are trained on one-dimensional, ordered text. So the authors ask: could two stages of “one-dimensional, causally ordered” reasoning, chained together, handle a two-dimensional image?\n\n- The first stage is in the encoder: rearrange the vision tokens into a reading order based on the image content.\n- The second stage is in the decoder: output text autoregressively in that order.\n\nThey use document reading as the testing ground, because documents have multi-column layouts, formulas, and tables, and their reading order carries a logic of its own.\n\n## Solving the Problem\n\n### Swapping CLIP for a Language Model\n\n[![Left: DeepEncoder, an 80M Tokenizer followed by CLIP ViT 300M; right: DeepEncoder V2, the same Tokenizer followed by a language model based on Qwen2 500M, with a set of learnable queries appended after the vision tokens; only the outputs of the queries are passed on](/content/papers/assets/e199e1e56c505533eb91beb5fa681e7a2366dac66b585a0317ab82b1c950d79d.png)](/content/papers/assets/e199e1e56c505533eb91beb5fa681e7a2366dac66b585a0317ab82b1c950d79d.png)\n\n*Figure 1 from the paper, excerpted from [**page 1 of the v1 PDF**](https://arxiv.org/pdf/2601.20552v1#page=1), CC BY 4.0. The previous DeepEncoder is on the left, DeepEncoder V2 on the right.*\n\nThe upper halves of both sides are the same: a Tokenizer of about 80 million parameters that reuses the previous generation's SAM-base plus two convolutional layers to compress the token count to one sixteenth.\n\nThe difference is in the lower half. The previous generation used a 300-million-parameter CLIP for global attention; V2 replaces it with a language-model architecture initialized from **Qwen2-0.5B** and appends a set of **learnable queries** after the vision tokens (the authors call them causal flow tokens).\n\nThere are as many queries as vision tokens. In the end, only the query half of the output goes to the decoder; the vision tokens themselves are not passed on.\n\nThe authors also tried another arrangement: put the vision tokens in a separate encoder and let the queries read them through cross-attention, an encoder–decoder structure like mBART's. Training failed to converge. Their guess is that, isolated in another encoder, the vision tokens didn't interact enough with the queries; only after the vision tokens became a prefix in the same sequence, where every layer can exchange information, did training work.\n\n### The Mask Decides Who Sees Whom\n\n[![Attention mask illustration: a ViT-style bidirectional mask and an LM-style causal mask combine into the DeepEncoder V2 mask; in the first three rows, the vision tokens see one another, and in the last three rows, the queries can see all the vision tokens and the queries before them](/content/papers/assets/2ba25df8f4e3579425551e93bdce7b8b0f180bb349cef46511dce7f44ffd68f3.png)](/content/papers/assets/2ba25df8f4e3579425551e93bdce7b8b0f180bb349cef46511dce7f44ffd68f3.png)\n\n*Figure 5 from the paper, excerpted from [**page 7 of the v1 PDF**](https://arxiv.org/pdf/2601.20552v1#page=7), CC BY 4.0. Green means attention is allowed, pink means it is blocked, and white circles mark the token itself.*\n\nThis figure is the heart of the design. Let's walk through it with three vision tokens $v_0,v_1,v_2$ and three queries $q_0,q_1,q_2$. Rows are “who is looking,” columns are “what is being looked at,” and 1 means attention is allowed:\n\n```text\n       v0 v1 v2 | q0 q1 q2\nv0      1  1  1 |  0  0  0\nv1      1  1  1 |  0  0  0\nv2      1  1  1 |  0  0  0\n----------------+---------\nq0      1  1  1 |  1  0  0\nq1      1  1  1 |  1  1  0\nq2      1  1  1 |  1  1  1\n```\n\n- **Top left**: the vision tokens all see one another. This is bidirectional attention, as in ViT, and it keeps CLIP's whole-image view.\n- **Top right**: the vision tokens can't see the queries, so the image features aren't affected by the order of the queries.\n- **Bottom left**: every query can see all the vision tokens.\n- **Bottom right**: among the queries, the mask is lower-triangular; $q_1$ sees only $q_0$ and itself, the same causal attention as in a decoder-only language model.\n\nThe paper writes the mask as:\n\n$$\nM=\\begin{bmatrix}\\mathbf{1}_{m\\times m} & \\mathbf{0}_{m\\times n}\\\\ \\mathbf{1}_{n\\times m} & \\mathrm{LowerTri}(n)\\end{bmatrix},\\quad n=m\n$$\n\nThe lower triangle in the bottom right is where the “order” comes from. $q_0$ decides first what to take from the image; $q_1$, knowing what $q_0$ took, decides what comes next; and so on. Lined up in a row, the outputs of the queries are themselves a reading order.\n\nThe authors write the whole process as:\n\n$$\n\\mathbf{O}=\\mathcal{D}\\big(\\pi_Q(\\mathcal{T}^L(\\mathcal{E}(\\mathbf{I})\\oplus\\mathbf{Q}_0;\\mathbf{M}))\\big)\n$$\n\n$\\mathcal{E}$ is the Tokenizer, $\\oplus$ concatenates the vision tokens and the queries, $\\mathcal{T}^L$ is an $L$-layer masked Transformer, $\\pi_Q$ keeps only the last $n$ outputs, that is, the outputs of the queries, and $\\mathcal{D}$ is the decoder.\n\n### How Many Tokens Does a Page Take?\n\nTo avoid preparing a separate set of queries for every resolution, the authors fix two views:\n\n- **Global view** at 1024×1024: 1024/16 = 64, and 64×64 = 4,096 patches; compressed sixteenfold, that is **256** tokens, paired with 256 queries.\n- **Local crops** at 768×768: 768/16 = 48, and 48×48 = 2,304 patches; after compression, **144** tokens, with every crop sharing the same set of 144 queries.\n\nThe number of crops $k$ ranges from 0 to 6, and there is no cropping when both sides are smaller than 768. So the decoder receives $144k+256$ tokens: at least 256 and at most $6\\times144+256=1{,}120$.\n\nThe authors make a point of comparing the upper limit: the previous generation's Gundam mode goes up to 1,156, and Gemini 3 Pro's vision-token cap is also 1,120.\n\nThe decoder is unchanged: it is still the previous generation's 3B MoE, activating about 500 million parameters.\n\n### Training\n\nThe data sources are the same as for the previous generation, with OCR data making up 80%. Two small adjustments: OCR 1.0 data is split by content into text, formulas, and tables and sampled at 3:1:1; and in the layout annotations, semantically similar categories are merged, such as figure captions and figure titles.\n\nTraining has three stages:\n\n1. **Encoder pretraining**: the Tokenizer is carried over from the previous generation, and the LM-style encoder is initialized from Qwen2-0.5B-base and paired with a small decoder for next-token prediction. 160 A100s, about 100 million image-text pairs.\n2. **Query enhancement**: the Tokenizer is frozen, and the encoder and the DeepSeek decoder are trained together to strengthen the queries' ability to reorder.\n3. **Decoder specialization**: the whole encoder is frozen and only the decoder is trained. At the same batch size, training runs more than twice as fast.\n\n## Discussion\n\n### OmniDocBench v1.5\n\nThe main evaluation is OmniDocBench v1.5: 1,355 pages across 9 document types, in both Chinese and English.\n\n| Model | Vision-token cap | Overall | Text ED | Reading-order ED |\n| --- | ---: | ---: | ---: | ---: |\n| DeepSeek-OCR (9 crops) | 1,156 | 87.36 | 0.073 | 0.085 |\n| DeepSeek-OCR 2 | 1,120 | 91.09 | 0.048 | 0.057 |\n\n*Table: Excerpted from Table 1 of the paper. ED is edit distance, where lower is better; for Overall, higher is better.*\n\nWith similar data sources, the overall score rises by 3.73, and the reading-order edit distance falls from 0.085 to 0.057. In Table 2, the authors also compare overall edit distance against Gemini 3 Pro: under the same 1,120-token cap, DeepSeek-OCR 2 scores 0.100 and Gemini 3 Pro 0.115.\n\nThe same Table 1 also includes two notable rivals. PaddleOCR-VL scores 92.86 and MinerU2.5 90.67; both do layout analysis first and then recognize cropped regions, and the paper puts them in the Pipeline category. In other words, on this benchmark, the end-to-end DeepSeek-OCR 2 has come very close to the two-stage systems but hasn't yet passed the best of them.\n\n### The Nine Document Types, One by One\n\nTable 3 breaks down both generations' results across the nine document types. Reading-order edit distance **improves in all nine categories**; for example, colorful textbooks go from 0.125 to 0.066, and notes from 0.089 to 0.035.\n\nText edit distance is less consistent. Most categories improve, but books go from 0.022 to 0.033 and newspapers from 0.131 to 0.139. For newspapers, the authors give two reasons: the lower vision-token cap isn't enough for extremely text-heavy newspaper pages, which more crops could address in the future; and there are only 250,000 newspaper training samples, too few for the new encoder.\n\n### What to Watch in Production\n\nThis section is short, but I think it is the most practical part of the whole paper.\n\nInside the company, DeepSeek-OCR serves two purposes: reading images and documents that users upload, on behalf of DeepSeek's language models, and batch-processing PDFs to produce pretraining data. Production has no ground truth to compare against, so the authors use the **repetition rate** as the primary observable quality metric:\n\n| Model | Online user images | Pretraining PDFs |\n| --- | ---: | ---: |\n| DeepSeek-OCR | 6.25% | 3.69% |\n| DeepSeek-OCR 2 | 4.17% | 2.88% |\n\n*Table: Table 4 of the paper; a lower repetition rate is better.*\n\nRepetition is the most easily observed failure of generative OCR: the same passage keeps coming out, and a program catches it at once.\n\nThe flip side is that without ground truth, errors of the “fluent but wrong” kind are invisible to the repetition rate. For applications that make decisions from OCR output, this table is a reminder that production monitoring needs separate methods that can see such errors, such as comparing against the text layer, checking field formats, or sampling outputs for human review.\n\n### The Authors' Vision and Later Results\n\nIn the discussion section, the authors think bigger: a causal encoder that reorders what it sees, followed by a causal decoder that does the task reasoning, might turn two stages of one-dimensional reasoning into “genuine 2D reasoning.” They also envision the same LM-style encoder handling images, audio, and text just by swapping in a different set of queries. For now, these are only prospects; the paper validates the design only on document reading.\n\nOmniDocBench was later revised to v1.6, adding 296 harder samples. On the official leaderboard, DeepSeek-OCR 2 scores 90.25; the top five are a group of specialized models of around one billion parameters, the best of them scoring about 97. [OmniDocBench README](https://github.com/opendatalab/OmniDocBench)\n\n## Conclusion\n\nDeepSeek-OCR 2's change is tightly focused: it replaces only the back half of the encoder, letting a set of learnable queries read the image in sequence and rearrange the vision tokens into reading order.\n\n1. **The mask design is the key.** The vision tokens see one another bidirectionally, the queries attend causally among themselves, and only the outputs of the queries are passed on.\n2. **Reading order improves across the board.** On OmniDocBench v1.5, reading-order edit distance falls from 0.085 to 0.057, and all nine document types improve.\n3. **Text recognition doesn't improve in every category.** Text edit distance for books and newspapers actually rises slightly; newspapers are held back by the token cap and the training data.\n4. **The production metric sees only repetition.** The repetition rate falls from 6.25% to 4.17%, but this metric can't see fluent misreadings.\n\nHanding reading order to a set of queries that look at the image in sequence is a very intuitive idea. It makes the model read documents more the way a person does; whether the characters it reads were seen clearly or filled in from context still has to be checked separately.\n","imageSizes":{"/content/papers/assets/e199e1e56c505533eb91beb5fa681e7a2366dac66b585a0317ab82b1c950d79d.png":{"width":1111,"height":588},"/content/papers/assets/2ba25df8f4e3579425551e93bdce7b8b0f180bb349cef46511dce7f44ffd68f3.png":{"width":1194,"height":483}}}