{"notes":[{"id":"text-spotting/paddleocr-vl","category":"text-spotting","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"PaddleOCR-VL","subtitle":"Layout First, Then Read","author":"Z. Yuan","paperDate":"2025-10-16","publishedOn":"2026-10-31","world":"external/paddleocr-vl","paperTitle":"PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model","paperUrl":"https://arxiv.org/abs/2510.14528","bibliography":{"title":"PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model","authors":["Cheng Cui","Ting Sun","Suyin Liang","Tingquan Gao","Zelun Zhang","Jiaxuan Liu","Xueqing Wang","Changda Zhou","Hongen Liu","Manhui Lin","Yue Zhang","Yubo Zhang","Handong Zheng","Jing Zhang","Jun Zhang","Yi Liu","Dianhai Yu","Yanjun Ma"],"url":"https://arxiv.org/abs/2510.14528","pdfUrl":"https://arxiv.org/pdf/2510.14528v4","codeUrl":"https://github.com/PaddlePaddle/PaddleOCR","preprint":{"id":"2510.14528","submittedOn":"2025-10-16","url":"https://arxiv.org/abs/2510.14528"},"publication":null,"citation":"@misc{textspottingpaddleocrvl2025,\n  title = {{PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model}},\n  author = {Cheng Cui and Ting Sun and Suyin Liang and Tingquan Gao and Zelun Zhang and Jiaxuan Liu and Xueqing Wang and Changda Zhou and Hongen Liu and Manhui Lin and Yue Zhang and Yubo Zhang and Handong Zheng and Jing Zhang and Jun Zhang and Yi Liu and Dianhai Yu and Yanjun Ma},\n  year = {2025},\n  eprint = {2510.14528},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2510.14528}\n}\n","citationPath":"/content/papers/citations/text-spotting/paddleocr-vl.c7969231dd04.bib"},"thumbnail":{"src":"/content/papers/assets/8ee9dd9a20b760af5b9fa7f40b97755487d35e7d6eab2577db214b267ad05ef7.png","width":1288,"height":555},"locale":"en","path":"/content/papers/en/text-spotting/paddleocr-vl.9b675c01776f.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"two-stages","text":"Two Stages","depth":3},{"id":"layout-detection-plus-ordering","text":"Layout: Detection Plus Ordering","depth":3},{"id":"recognition-a-small-09b-model","text":"Recognition: A Small 0.9B Model","depth":3},{"id":"training-and-data","text":"Training and Data","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"omnidocbench-v15","text":"OmniDocBench v1.5","depth":3},{"id":"speed","text":"Speed","depth":3},{"id":"later-versions","text":"Later Versions","depth":3},{"id":"what-to-know-when-reading-this-paper","text":"What to Know When Reading This Paper","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[]}],"text":"By 2025, document parsing had split into roughly two paths.\n\nOne is the **pipeline**: layout detection, text recognition, table recognition, and formula recognition each get a specialized model, and the models are chained together. It is accurate, but there are many parts and many seams, and errors propagate all the way down.\n\nThe other is **end-to-end**: one VLM looks at the whole page image and writes out Markdown directly. The workflow is simple, but a single page means generating thousands of tokens, which is slow and memory-hungry; on multi-column pages or pages that mix text and images, the reading order often gets scrambled, and the model may produce content that isn't in the original.\n\nPaddleOCR-VL, from Baidu's PaddlePaddle team, takes a middle path: **a dedicated detection model handles the layout, and a very small VLM handles the content of each block**. It was uploaded to arXiv on October 16, 2025; this note follows v4 from November 25. The code and models are open source.\n\n## Defining the Problem\n\nThe authors are blunt about both paths.\n\nPipeline systems are complex to integrate, accumulate errors, and hit a ceiling on very complex documents. End-to-end models rely on very long autoregressive generation, with high latency and memory use; on multi-column or mixed text-and-image pages, their layout analysis tends to be unstable and prone to hallucination.\n\nThey wanted a system that could handle many languages and element types, yet was small enough to deploy in practice.\n\n## Solving the Problem\n\n### Two Stages\n\n[![PaddleOCR-VL overview: on the left, unstructured documents such as PDF, PNG, and JPG; in the middle, PP-DocLayoutV2 first performs layout analysis and outputs block positions, categories, and reading order, then passes the blocks in batches to PaddleOCR-VL-0.9B, which recognizes them with four instructions: OCR, table, formula, and chart; on the right, Markdown and JSON output](/content/papers/assets/8ee9dd9a20b760af5b9fa7f40b97755487d35e7d6eab2577db214b267ad05ef7.png)](/content/papers/assets/8ee9dd9a20b760af5b9fa7f40b97755487d35e7d6eab2577db214b267ad05ef7.png)\n\n*Figure 2 from the paper, excerpted from [**page 5 of the v4 PDF**](https://arxiv.org/pdf/2510.14528v4#page=5), CC BY 4.0.*\n\nReading from left to right:\n\n1. **PP-DocLayoutV2** reads the whole page image, finds the position and category of each block (text, table, formula, chart, and so on), and predicts their reading order.\n2. Each block is cropped out and passed in batches to **PaddleOCR-VL-0.9B**, together with the instruction that matches its category: OCR, table recognition, formula recognition, or chart recognition.\n3. Lightweight post-processing assembles the results of the two stages and outputs Markdown and JSON.\n\n### Layout: Detection Plus Ordering\n\nPP-DocLayoutV2 chains two networks:\n\n- A detection model based on **RT-DETR**, which finds and classifies layout elements.\n- A 6-layer Transformer **pointer network**, which works out the reading order.\n\nThe ordering method is worth a closer look. Each detected block is represented by an absolute 2D positional encoding plus a category embedding; the attention also adds Relation-DETR's geometric bias, so the model directly takes into account the relative position of any two blocks. Next, a pairwise relation head computes a “which should be read first” score for every pair of blocks, giving an $N\\times N$ matrix. Finally, a deterministic “win-accumulation” decoding produces a consistent order.\n\nTry it on a three-block page: a title on top, with two columns below it. Each number in the table is the probability that “the block in this row (left) should be read before the block in this column (top)” (illustrative values):\n\n| | Title | Left column | Right column |\n| --- | --- | --- | --- |\n| Title | — | 0.92 | 0.95 |\n| Left column | | — | 0.81 |\n\nCompare each pair once: the title beats both the left and the right column, two wins; the left column beats the right column, one win; the right column has zero wins. Sorting by wins gives title → left column → right column.\n\nThe advantage is that the order comes from comparing all the pairs, so a single wrong pairwise judgment can't throw it off much; and it is a separate small model that can be trained and evaluated on its own.\n\n### Recognition: A Small 0.9B Model\n\nPaddleOCR-VL-0.9B uses a LLaVA-style architecture with three parts:\n\n- **Vision encoder**: NaViT-style, supporting native-resolution input, initialized from Keye-VL's vision model. The authors say that because images aren't distorted by resizing or tiling, there are fewer hallucinations, and text-dense content is handled better.\n- **Projector**: a two-layer MLP that merges neighboring visual features before sending them to the language model.\n- **Language model**: **ERNIE-4.5-0.3B**, with 3D-RoPE added. The authors explain specifically why they chose such a small language model: autoregressive generation emits one token at a time, so the decoder's size directly determines latency.\n\nThe whole model supports 109 languages.\n\n### Training and Data\n\nTraining has two stages:\n\n1. **Alignment pretraining**: 29 million image-text pairs for one epoch, with a maximum resolution of 1280×28×28.\n2. **Instruction fine-tuning**: 2.7 million samples for two epochs, with the maximum resolution raised to 2048×28×28. There are four tasks: OCR, tables (output in OTSL format), formulas (LaTeX that distinguishes inline from display formulas), and charts (output as Markdown tables).\n\nThe data comes from public datasets, synthetic data, documents of all kinds collected from the web, and internal data the team has accumulated over many years. The annotation process deserves a look:\n\n1. First, the team's own pipeline tool, PP-StructureV3, produces preliminary pseudo-labels.\n2. The original image and the pseudo-labels are then handed together to large multimodal models such as ERNIE-4.5-VL and Qwen2.5-VL, which correct them.\n3. Finally, a **hallucination filtering** step removes content the large models may have fabricated.\n\nEven when building training data, you have to guard against large models filling in text.\n\nThey also built an evaluation engine that sorts text into 23 categories, such as Chinese, English, printed, handwritten, Japanese, Latin, and emoji, and tables into 20; they use it to find hard samples and then add targeted data.\n\n## Discussion\n\n### OmniDocBench v1.5\n\nOn OmniDocBench v1.5 (1,355 pages), PaddleOCR-VL scores 92.86 overall, with MinerU2.5 second at 90.67. It also leads on every sub-metric: text edit distance 0.035, formula CDM 91.22, table TEDS 90.89, and reading-order edit distance 0.043.\n\nOn olmOCR-Bench, its overall score is 80.0, with the top results in the ArXiv and headers-and-footers categories.\n\nThese are all results reported by the authors themselves. When the DeepSeek-OCR 2 paper later cited official OmniDocBench results, PaddleOCR-VL again appeared at 92.86, ranked ahead of every other model.\n\n### Speed\n\nOn a single A100, with batches of 512 PDFs, the authors measured end-to-end time from input PDF paths to complete Markdown. Deployed with FastDeploy, PaddleOCR-VL processes 53.1% more pages per second than MinerU2.5 and outputs 50.9% more tokens per second.\n\nA small language model, plus recognizing only the cropped blocks, gives it a considerable edge in speed.\n\n### Later Versions\n\nPaddleOCR-VL was followed by version 1.5 (January 2026) and version 1.6 (June 2026). On the official OmniDocBench v1.6 leaderboard, PaddleOCR-VL-1.6 scores 96.34 overall, among the top entries. [OmniDocBench README](https://github.com/opendatalab/OmniDocBench)\n\n### What to Know When Reading This Paper\n\n1. **Most results are self-reported.** The OmniDocBench results can later be checked against the official leaderboard; the other in-house evaluations can't be independently verified.\n2. **If the layout is wrong, the second stage can't recover.** The two-stage design concentrates risk in layout analysis: if a block is missed in detection, its text is never recognized. The detection part of PP-DocLayoutV2 is trained on over 20,000 in-house samples; the detector is then frozen and the ordering network trained on its own.\n3. **Large models took part in annotation.** The pseudo-labels were corrected by ERNIE-4.5-VL and Qwen2.5-VL and then passed through hallucination filtering; the paper gives no numbers on how much the filter catches.\n\n## Conclusion\n\nPaddleOCR-VL's design is a very pragmatic division of labor:\n\n1. **Layout goes to a detection model**: RT-DETR finds the blocks, and a pointer network orders them through pairwise comparison.\n2. **Content goes to a very small VLM**: a NaViT vision encoder plus a 0.3B ERNIE, with one instruction for each of the four element types.\n3. **Both scores and speed**: 92.86 overall on OmniDocBench v1.5, and about 50% higher throughput than MinerU2.5.\n\nLooking back from the end-to-end trend, this design is like inviting the pipeline back, only with the character-reading component replaced by a small VLM. It reads one block at a time and its outputs are short, which, the authors argue, lowers both latency and the risk of hallucination.\n","imageSizes":{"/content/papers/assets/8ee9dd9a20b760af5b9fa7f40b97755487d35e7d6eab2577db214b267ad05ef7.png":{"width":1288,"height":555}}}