{"notes":[{"id":"model-evaluation/linguistic-crutch","category":"model-evaluation","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Linguistic Crutch","subtitle":"The Price of Filling In the Blanks","author":"Z. Yuan","paperDate":"2026-01-07","publishedOn":"2026-02-21","paperTitle":"Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR","paperUrl":"https://arxiv.org/abs/2601.03714","bibliography":{"title":"Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR","authors":["Yunhao Liang","Ruixuan Ying","Bo Li","Hong Li","Kai Yan","Qingwen Li","Min Yang","Okamoto Satoshi","Zhe Cui","Shiwen Ni"],"url":"https://arxiv.org/abs/2601.03714","pdfUrl":"https://arxiv.org/pdf/2601.03714v2","codeUrl":null,"preprint":{"id":"2601.03714","submittedOn":"2026-01-07","url":"https://arxiv.org/abs/2601.03714"},"publication":null,"citation":"@misc{modelevaluationlinguisticcrutch2026,\n  title = {{Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR}},\n  author = {Yunhao Liang and Ruixuan Ying and Bo Li and Hong Li and Kai Yan and Qingwen Li and Min Yang and Okamoto Satoshi and Zhe Cui and Shiwen Ni},\n  year = {2026},\n  eprint = {2601.03714},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2601.03714}\n}\n","citationPath":"/content/papers/citations/model-evaluation/linguistic-crutch.af1f73a098b5.bib"},"thumbnail":{"src":"/content/papers/assets/3dfc834dee089d76acab6a1ce767b06c592ef84c6b48c2a5f79a7723f9cc3f09.png","width":644,"height":300},"locale":"en","path":"/content/papers/en/model-evaluation/linguistic-crutch.41a291e10181.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"keeping-the-image-conditions-identical","text":"Keeping the Image Conditions Identical","depth":3},{"id":"three-kinds-of-disruption","text":"Three Kinds of Disruption","depth":3},{"id":"two-more-tests","text":"Two More Tests","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"when-sentences-stop-making-sense","text":"When Sentences Stop Making Sense","depth":3},{"id":"when-words-stop-looking-like-words","text":"When Words Stop Looking Like Words","depth":3},{"id":"what-about-other-models","text":"What About Other Models?","depth":3},{"id":"question-answering-and-long-context","text":"Question Answering and Long Context","depth":3},{"id":"what-to-know-when-reading-this-paper","text":"What to Know When Reading This Paper","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"deepseek/deepseek-ocr","relation":"citing","topic":""}]},{"id":"deepseek/deepseek-ocr","category":"deepseek","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"DeepSeek-OCR","subtitle":"A Picture Is Worth a Thousand Words","author":"Z. Yuan","paperDate":"2025-10-21","publishedOn":"2026-03-09","world":"external/deepseek-ocr","paperTitle":"DeepSeek-OCR: Contexts Optical Compression","paperUrl":"https://arxiv.org/abs/2510.18234","bibliography":{"title":"DeepSeek-OCR: Contexts Optical Compression","authors":["Haoran Wei","Yaofeng Sun","Yukun Li"],"url":"https://arxiv.org/abs/2510.18234","pdfUrl":"https://arxiv.org/pdf/2510.18234v1","codeUrl":"https://github.com/deepseek-ai/DeepSeek-OCR","preprint":{"id":"2510.18234","submittedOn":"2025-10-21","url":"https://arxiv.org/abs/2510.18234"},"publication":null,"citation":"@misc{deepseekdeepseekocr2025,\n  title = {{DeepSeek-OCR: Contexts Optical Compression}},\n  author = {Haoran Wei and Yaofeng Sun and Yukun Li},\n  year = {2025},\n  eprint = {2510.18234},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2510.18234}\n}\n","citationPath":"/content/papers/citations/deepseek/deepseek-ocr.66e8e5a7db8d.bib"},"thumbnail":{"src":"/content/papers/assets/6f53ecfe63f39804454a69ef4468c93e8018fa1afbe08d1bb52b7e8d52d194c3.png","width":1288,"height":350},"locale":"en","path":"/content/papers/en/deepseek/deepseek-ocr.37f4dfc7859e.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"overall-architecture","text":"Overall Architecture","depth":3},{"id":"walking-one-10241024-page-through","text":"Walking One 1024×1024 Page Through","depth":3},{"id":"the-decoder-recovering-text-from-a-compressed-representation","text":"The Decoder: Recovering Text from a Compressed Representation","depth":3},{"id":"data-and-training","text":"Data and Training","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"compression-experiments-100-pages-from-fox","text":"Compression Experiments: 100 Pages from Fox","depth":3},{"id":"real-documents-omnidocbench","text":"Real Documents: OmniDocBench","depth":3},{"id":"optical-compression-and-forgetting","text":"Optical Compression and Forgetting","depth":3},{"id":"discussion-after-release","text":"Discussion After Release","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"model-evaluation/linguistic-crutch","relation":"referenced","topic":""}]}],"text":"DeepSeek-OCR, released in October 2025, claimed that as long as a page of English has no more than 10 times as many text tokens as vision tokens, it can be read back with about 97% precision; at 20× compression, precision is still about 60%. That led many people to hope that images might be used to compress an LLM's long context.\n\nThe authors of this paper ask a more basic question: how much of those scores actually comes from the image?\n\nThe title puts it plainly: visual merit, or linguistic crutch? DeepSeek-OCR's decoder is a language model, and it already knows how English is usually written. When the vision tokens aren't clear enough, it may well fill in the missing letters from the surrounding text. Call it **filling in the blanks**. Fill them in correctly, and the score goes up.\n\nThe paper comes from researchers at the Chengdu Institute of Computer Applications of the Chinese Academy of Sciences, Tohoku University in Japan, and other institutions. It was uploaded to arXiv on January 7, 2026; this note follows v2, updated the next day. It is currently a preprint.\n\n\n> [!TIP]\n> \n> **What Is a Language Prior?**\n> \n> A prior is an expectation you hold before seeing the evidence. A language model has read enormous amounts of text: it knows that “has already been” is often followed by a past participle, and that “Speaker” is more common than “Squeaker.”\n> \n> When reading text, the model uses two kinds of clues at once: the strokes in the image, and this expectation of “what usually comes here.” This paper sets out to measure the two separately.\n\n\n## Defining the Problem\n\nThe most direct way to measure the language prior's contribution is to take away the regularities of language, keep every other condition of the image the same, and see how far the score drops.\n\nThe authors pose five research questions:\n\n1. How does sentence-level semantic disruption affect DeepSeek-OCR?\n2. What about word-level disruption, including swapped letters, fully scrambled words, and completely random strings?\n3. Is this reliance unique to DeepSeek-OCR, or do other OCR systems and VLMs share it?\n4. How much meaning does DeepSeek-OCR retain for question answering and visual question answering?\n5. At what context length does optical compression break down?\n\n## Solving the Problem\n\n### Keeping the Image Conditions Identical\n\nThe benchmark is Fox's 112 English document pages, with ground truths between 600 and 2,500 tokens long.\n\nThe authors don't compare the original scanned pages directly with perturbed ones, since the two differ in font and layout. Instead, they first re-render the ground truth as clean images, called **text2png**, to serve as the baseline, and render every perturbed version through the same pipeline. That way, the only difference in the images is the arrangement of the letters.\n\n### Three Kinds of Disruption\n\n**Sentence level**: key nouns, verbs, and phrases are replaced with words that are spelled like English but make no sense. For example:\n\n> has already been published in other issues of the Speaker, it won't all be repeated here.\n\nbecomes:\n\n> has already been breadboxed in other whispers of the Squeaker, it won't all be rebound here.\n\nThe letterforms and layout are all still there; the meaning of the sentence is gone.\n\n**Word level**: 5% or 10% of the words are chosen at random.\n\n- **Swap**: two letters within a word are exchanged, so already becomes ayreadl. The word's structure is mostly intact, and language knowledge can repair it.\n- **Shuffle**: all the letters of a word are scrambled, so repeated becomes eepetadr. The letter distribution is the same, but the structure is destroyed.\n\n**Zero prior**: the whole document is replaced with “words” of 2 to 10 random upper- and lowercase letters, with the same length distribution as the original. There is no vocabulary or grammar to lean on at all; the authors call it a “truth serum” for OCR systems.\n\n### Two More Tests\n\n- **Question answering**: the authors annotate each Fox page with three factual multiple-choice questions and compare DeepSeek-OCR answering from the image (VQA), language models answering from the text itself (QA), and other VLMs with 3B to 8B parameters.\n- **Context length**: GPT-5.1 is asked to write five English stories of about 5,000 words each. They are concatenated repeatedly to about 20,000 tokens, then cut into 40 segments from 500 to 20,000 tokens in steps of 500, rendered as document images, and read by DeepSeek-OCR in its four modes.\n\n## Discussion\n\n### When Sentences Stop Making Sense\n\n[![Top: an ordinary sentence, which DeepSeek-OCR reads correctly; bottom: a sentence with words replaced by breadboxed, whispers, Squeaker, and rebound, for which the model outputs common words such as rebroadcast and Square](/content/papers/assets/3dfc834dee089d76acab6a1ce767b06c592ef84c6b48c2a5f79a7723f9cc3f09.png)](/content/papers/assets/3dfc834dee089d76acab6a1ce767b06c592ef84c6b48c2a5f79a7723f9cc3f09.png)\n\n*Figure 1 from the paper, excerpted from [**page 1 of the v2 PDF**](https://arxiv.org/pdf/2601.03714v2#page=1). The top shows normal input and the bottom input after semantic disruption; on the right, the words the model rewrote are highlighted in yellow.*\n\nLook at the bottom row first. The image clearly shows Squeaker, and the model outputs Square; breadboxed and rebound both come out as rebroadcast. Every one of these outputs is a real English word, and the sentence reads more smoothly than the original.\n\nHere are the average precisions over the 112 pages:\n\n| Mode (vision tokens) | text2png | Sentence disruption | Drop |\n| --- | ---: | ---: | ---: |\n| Tiny (64) | 88.00 | 76.75 | 11.25 |\n| Small (100) | 95.23 | 91.56 | 3.67 |\n| Base (256) | 97.93 | 97.31 | 0.62 |\n\n*Table: Averages (%) from Table 1 of the paper.*\n\nThe fewer the vision tokens, the bigger the drop. With 256 tokens, whether the sentence makes sense barely affects the result; with only 64, precision falls by more than 11 percentage points.\n\n### When Words Stop Looking Like Words\n\nSwap and Shuffle point the same way. With 10% of the words disrupted, Tiny mode drops by an average of 9.89 and 11.30 percentage points, respectively. Shuffle hurts more than Swap, which the authors take as a sign that the model relies on normal letter order to decode.\n\nIndividual cases make it even clearer:\n\n| Case | Input | Output |\n| --- | --- | --- |\n| Swap | has ayreadl | has already |\n| Shuffle | it won't all be eepetadr here. | I won't be all expected here. |\n| Zero prior | EYuoV qUtjpy pWxZCks | E'vuol u'qtippy piwZckus |\n\n*Table: Excerpted from Table 5 of the paper.*\n\nIn the first two rows, the model acts like an autocorrector, turning the odd words into fluent English. In the last row, there is no direction to “correct” toward, so the model forces the random letters into something that looks like syllables; the result is neither the original nor a word.\n\nAverage precision with zero prior: Tiny 19.84%, Small 42.12%, Base 61.70%. Compared with text2png, Tiny loses 68 percentage points, and even Base loses 36.\n\n### What About Other Models?\n\n[![Two tables: precision by text-length range on normal text and on random-letter text for end-to-end models such as DeepSeek-OCR, dots.ocr, Qwen2.5-VL, GOT-OCR, MonkeyOCR, SmolDocling, Nougat, and HunyuanOCR, and for two traditional pipelines, MinerU and PaddleOCR-v5](/content/papers/assets/5d2501d5e3e771beeb643c417a2fd5186c6c1994a4caa911b27ca6896df3907f.png)](/content/papers/assets/5d2501d5e3e771beeb643c417a2fd5186c6c1994a4caa911b27ca6896df3907f.png)\n\n*Tables 6–7 from the paper, excerpted from [**page 6 of the v2 PDF**](https://arxiv.org/pdf/2601.03714v2#page=6). The upper table is normal text and the lower random letters; the two rightmost columns are traditional pipelines.*\n\nThe bottom row is the average. Here are a few for comparison:\n\n| Model | Normal text | Random letters |\n| --- | ---: | ---: |\n| PaddleOCR-v5 (pipeline) | 94.44 | 89.53 |\n| MonkeyOCR 3B | 97.37 | 79.97 |\n| GOT-OCR (0.58B) | 98.00 | 57.71 |\n| Qwen2.5-VL 72B | 98.69 | 56.78 |\n| Nougat (0.35B) | 96.82 | 37.94 |\n| DeepSeek-OCR Tiny | 88.00 | 19.84 |\n\nOn normal text, the end-to-end models are almost all above 94%, with no visible difference between them. With random letters, they drop anywhere from 17 to 68 percentage points, while PaddleOCR-v5, which first detects text lines and then recognizes them one by one, drops only 4.9.\n\nThe authors' explanation is that a pipeline separates visual recognition from language decoding, so it can still read text that has no linguistic regularity. End-to-end models generate text directly from the image, and wherever the visual evidence is uncertain, language knowledge fills in the rest.\n\nThe other pipeline, MinerU, scores below 10% on both kinds of text, because its layout detection treats the whole image as one block, leaving the recognition stage nothing to work with. This failure has nothing to do with the language prior and should be left out of the comparison.\n\n### Question Answering and Long Context\n\nIn the QA test, DeepSeek-OCR answering from the image is about 20% accurate, while random guessing on four-option questions would score 25%; language models such as Qwen2.5-3B, given the text directly, exceed 90%. From this the authors conclude that optical compression destroys the semantics needed for reasoning.\n\nThat inference should be taken with a grain of salt. The paper also tested giving the **ground-truth text** directly to DeepSeek-OCR's decoder, and accuracy was only 27.7%. The original DeepSeek-OCR paper is explicit that the model had no supervised fine-tuning, is not a chat model, and needs completion-style prompts to bring out some of its abilities. So a sizable share of the low QA scores may come from a decoder that was never good at answering questions in the first place, and it can't all be blamed on compression.\n\n[![OCR precision of DeepSeek-OCR's four modes at different context lengths, with one-standard-deviation bands; the horizontal axis is token count (thousands); Tiny, Base, and Large drop to zero at about 8.5K, and Small mode lasts until about 10.5K](/content/papers/assets/eab689713edeb816901e8cb5dc077a8708b983866058f104214ba5fd1d9973c3.png)](/content/papers/assets/eab689713edeb816901e8cb5dc077a8708b983866058f104214ba5fd1d9973c3.png)\n\n*Figure 3 from the paper, excerpted from [**page 8 of the v2 PDF**](https://arxiv.org/pdf/2601.03714v2#page=8). The horizontal axis is the number of text tokens in one image (thousands); the vertical axis is precision.*\n\nThe long-context curves start falling earlier than the “collapse at around 10,000 tokens” in the authors' abstract: all four modes are already clearly lower between 1,000 and 3,000 tokens; Tiny, Base, and Large hit zero at about 8,500 tokens, while Small holds out until about 10,500 before it too drops to zero.\n\nThe paper gives examples from Base mode. At 2,500 tokens the output is nearly perfect, except that the name Xingfu is read as Yingxiu; by 3,000 tokens, the first half of the sentence has nothing to do with the original; at 3,500 and 4,000 tokens, it starts repeating the same phrase; at 8,000 tokens, it outputs a single sentence: “The following text is a placeholder for an image.” Beyond that, only empty HTML table tags remain.\n\nThis experiment, however, crams more and more text into **a single image**. Ten thousand tokens of English inside 1024×1024 leaves each character a pitifully small number of pixels, so part of the collapse may simply be text too small to see. The authors read the result as optical compression making the long-context bottleneck worse; in our view, what this test answers more directly is “how much text one fixed-resolution image can hold.”\n\n### What to Know When Reading This Paper\n\n1. **Only dense English paragraphs were tested.** The authors state in their limitations section that they didn't test at scale on content such as formulas, code, and tables, where the language prior helps less to begin with.\n2. **Zero prior is an extreme condition.** Real documents rarely consist of whole pages of random letters. It suits measuring “how much can be read from the image itself,” and its numbers shouldn't be converted directly into error rates for everyday documents.\n3. **The QA and long-context tests both have confounding factors**, as discussed in the previous section.\n4. **The number of baselines.** The abstract says 13 baselines were compared; besides the two DeepSeek-OCR modes, the tables show 11 models or sizes.\n\n## Conclusion\n\nWith a very simple controlled experiment, this paper separates what was “seen” from what was “guessed” in OCR scores:\n\n1. **The fewer the vision tokens, the heavier the reliance on the language prior.** When sentences lose their meaning, Tiny drops 11 percentage points, while Base is barely affected.\n2. **Take away all linguistic regularity, and end-to-end models fall sharply across the board.** DeepSeek-OCR Tiny drops from 88% to about 20%, and Qwen2.5-VL 72B from 98.7% to 56.8%.\n3. **Traditional pipeline OCR is the most stable.** PaddleOCR-v5 drops only 4.9 percentage points.\n\nOptical compression works on normal English partly because English has regularities the decoder can use to fill things back in. There is nothing wrong with that in itself; people do the same when reading blurry text. The problem is that users can't tell which characters were filled in.\n\nFor engineers, the most useful thing in this paper is its testing method: **take the meaning away and test again**. Before using a VLM to read irregular text such as ID numbers, names, and codes, test it on random strings first. That is more reliable than checking a leaderboard.\n","imageSizes":{"/content/papers/assets/3dfc834dee089d76acab6a1ce767b06c592ef84c6b48c2a5f79a7723f9cc3f09.png":{"width":644,"height":300},"/content/papers/assets/5d2501d5e3e771beeb643c417a2fd5186c6c1994a4caa911b27ca6896df3907f.png":{"width":1277,"height":1100},"/content/papers/assets/eab689713edeb816901e8cb5dc077a8708b983866058f104214ba5fd1d9973c3.png":{"width":661,"height":466}}}