{"notes":[{"id":"text-recognition/diffusionstr","category":"text-recognition","legacyPath":"text-recognition/2306-diffusionstr","legacySlug":"2306-diffusionstr","legacyPaths":[],"title":"DiffusionSTR","subtitle":"噪声中的文字讯号","author":"Z. Yuan","paperDate":"2023-06-29","publishedOn":"2024-10-13","paperTitle":"DiffusionSTR: Diffusion Model for Scene Text Recognition","paperUrl":"https://arxiv.org/abs/2306.16707","bibliography":{"title":"DiffusionSTR: Diffusion Model for Scene Text Recognition","authors":["Masato Fujitake"],"url":"https://arxiv.org/abs/2306.16707","pdfUrl":"https://arxiv.org/pdf/2306.16707","codeUrl":null,"preprint":{"id":"2306.16707","submittedOn":"2023-06-29","url":"https://arxiv.org/abs/2306.16707"},"publication":{"venue":"ICIP","year":2023,"kind":"conference","url":"https://arxiv.org/abs/2306.16707"},"citation":"@misc{textrecognitiondiffusionstr2023,\n  title = {{DiffusionSTR: Diffusion Model for Scene Text Recognition}},\n  author = {Masato Fujitake},\n  year = {2023},\n  eprint = {2306.16707},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2306.16707}\n}\n","citationPath":"/content/papers/citations/text-recognition/diffusionstr.e8ee4c2db44d.bib"},"thumbnail":{"src":"/content/papers/assets/fe0f3bb5164b182f59bf735f76d6487408377e22beebbfe6f11bf0f6caba5afd.jpg","width":1464,"height":432},"locale":"zh-hans","path":"/content/papers/zh-hans/text-recognition/diffusionstr.0bd181a31eb7.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"transformer-架构","text":"Transformer 架构","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"与其他方法的比较","text":"与其他方法的比较","depth":3},{"id":"扩散模型的优势","text":"扩散模型的优势","depth":3},{"id":"字符感知头部影响","text":"字符感知头部影响","depth":3},{"id":"扩散步骤影响","text":"扩散步骤影响","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"text-spotting/got","relation":"comparison","topic":"生成式 OCR 的不同做法"},{"id":"text-recognition/dtrocr","relation":"comparison","topic":"生成式 OCR 的不同做法"}]},{"id":"text-spotting/got","category":"text-spotting","legacyPath":"text-spotting/2409-got","legacySlug":"2409-got","legacyPaths":[],"title":"GOT","subtitle":"包山包海的 OCR","author":"Z. Yuan","paperDate":"2024-09-03","publishedOn":"2024-10-06","paperTitle":"General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model","paperUrl":"https://arxiv.org/abs/2409.01704","bibliography":{"title":"General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model","authors":["Haoran Wei","Chenglong Liu","Jinyue Chen","Jia Wang","Lingyu Kong","Yanming Xu","Zheng Ge","Liang Zhao","Jianjian Sun","Yuang Peng","Chunrui Han","Xiangyu Zhang"],"url":"https://arxiv.org/abs/2409.01704","pdfUrl":"https://arxiv.org/pdf/2409.01704","codeUrl":null,"preprint":{"id":"2409.01704","submittedOn":"2024-09-03","url":"https://arxiv.org/abs/2409.01704"},"publication":null,"citation":"@misc{textspottinggot2024,\n  title = {{General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model}},\n  author = {Haoran Wei and Chenglong Liu and Jinyue Chen and Jia Wang and Lingyu Kong and Yanming Xu and Zheng Ge and Liang Zhao and Jianjian Sun and Yuang Peng and Chunrui Han and Xiangyu Zhang},\n  year = {2024},\n  eprint = {2409.01704},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2409.01704}\n}\n","citationPath":"/content/papers/citations/text-spotting/got.a6192bad407a.bib"},"thumbnail":{"src":"/content/papers/assets/7ed02fbacb92b589ea5efc9c02a180ddc03e3af515428f8db5b3a64ab0b754dd.jpg","width":1224,"height":908},"locale":"zh-hans","path":"/content/papers/zh-hans/text-spotting/got.7ea5f1407c05.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"第一阶段训练","text":"第一阶段训练","depth":3},{"id":"第二阶段训练","text":"第二阶段训练","depth":3},{"id":"第三阶段训练","text":"第三阶段训练","depth":3},{"id":"训练策略","text":"训练策略","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"纯文本文档-ocr-性能","text":"纯文本文档 OCR 性能","depth":3},{"id":"场景文本-ocr-性能","text":"场景文本 OCR 性能","depth":3},{"id":"格式化文档-ocr-性能","text":"格式化文档 OCR 性能","depth":3},{"id":"细粒度-ocr-性能","text":"细粒度 OCR 性能","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"text-recognition/dtrocr","relation":"comparison","topic":"生成式 OCR 的不同做法"},{"id":"text-recognition/diffusionstr","relation":"comparison","topic":"生成式 OCR 的不同做法"}]},{"id":"text-recognition/dtrocr","category":"text-recognition","legacyPath":"text-recognition/2308-dtrocr","legacySlug":"2308-dtrocr","legacyPaths":[],"title":"DTrOCR","subtitle":"解码器就好","author":"Z. Yuan","paperDate":"2023-08-30","publishedOn":"2024-11-15","paperTitle":"DTrOCR: Decoder-only Transformer for Optical Character Recognition","paperUrl":"https://arxiv.org/abs/2308.15996","bibliography":{"title":"DTrOCR: Decoder-only Transformer for Optical Character Recognition","authors":["Masato Fujitake"],"url":"https://arxiv.org/abs/2308.15996","pdfUrl":"https://arxiv.org/pdf/2308.15996","codeUrl":null,"preprint":{"id":"2308.15996","submittedOn":"2023-08-30","url":"https://arxiv.org/abs/2308.15996"},"publication":{"venue":"WACV","year":2024,"kind":"conference","url":"https://arxiv.org/abs/2308.15996"},"citation":"@misc{textrecognitiondtrocr2023,\n  title = {{DTrOCR: Decoder-only Transformer for Optical Character Recognition}},\n  author = {Masato Fujitake},\n  year = {2023},\n  eprint = {2308.15996},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2308.15996}\n}\n","citationPath":"/content/papers/citations/text-recognition/dtrocr.dcb015457224.bib"},"thumbnail":{"src":"/content/papers/assets/192548a108c1789007d9a90f8ecb479c783d1b198a507d1742a4c8e5cbadc81a.jpg","width":1224,"height":660},"locale":"zh-hans","path":"/content/papers/zh-hans/text-recognition/dtrocr.79d88c124d2d.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"合成数据预训练","text":"合成数据预训练","depth":3},{"id":"真实数据微调","text":"真实数据微调","depth":3},{"id":"实作细节","text":"实作细节","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"str-性能比较","text":"STR 性能比较","depth":3},{"id":"sroie-性能比较","text":"SROIE 性能比较","depth":3},{"id":"手写文本识别性能","text":"手写文本识别性能","depth":3},{"id":"中文识别性能","text":"中文识别性能","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"text-recognition/trocr","relation":"background","topic":"文字识别 Transformer 的预训练"},{"id":"text-spotting/got","relation":"comparison","topic":"生成式 OCR 的不同做法"},{"id":"text-recognition/diffusionstr","relation":"comparison","topic":"生成式 OCR 的不同做法"}]}],"text":"做研究的逻辑有时候很简单：\n\n看到别的领域有什么好东西，拿过来试试看，或许就是一篇论文。\n\n## 定义问题\n\n扩散模型已经流行好几年了，主要的应用还是集中在图像生成领域上。\n\n这么新鲜的东西，不如...\n\n拿来做文字辨识吧！\n\n## 解决问题\n\n### 模型架构\n\n![模型架构](/content/papers/assets/fe0f3bb5164b182f59bf735f76d6487408377e22beebbfe6f11bf0f6caba5afd.jpg)\n\n扩散模型的概念可以讲个三天三夜，这里就不多说了，我们假设你已经是扩散模型的高手。\n\n作者这里引入扩散模型，用来作为文字辨识的基底。\n\n整个架构的设计非常简单，首先将一个图像使用 ViT 进行编码，得到图像的特征表示。\n\n接着，以噪声填充的序列$x_T$作为输入，通过 Transformer 在视觉特征的条件下，生成逐步精炼的序列$x_{T-1}$，此过程重复$T$次，最终输出序列$x_0$。\n\n最后，$x_0$经由 FFN 转换为辨识的文字序列，并且字符感知头部负责预测字符的位置。\n\n扩散模型包含前向和逆向过程：\n\n- **前向过程**：原始扩散模型将数据点 $x_0$ 逐步加入噪声至高斯噪声 $x_T$；这篇处理离散字符序列，改用多项式扩散模型。\n- **逆向过程**：通过逐步去噪重建原始数据 $x_0$。\n\n作者使用了多项扩散模型来处理字符这类离散变量，并引入了特殊标记，如[EOS]（文字结束）、[PAD]（填充）、[MASK]（噪声）等。\n\n训练时使用简化的损失函数$L_{\\text{simple}}$，并采用均方误差损失进行稳定训练。\n\n除此之外，没有使用任何语言模型。因为作者认为语言模型的分支会让模型架构更为复杂，这样难以直观的看出扩散模型的效果。\n\n### Transformer 架构\n\n这里作者使用带有时间位置编码的 Transformer 解码器来转换扩散模型中的序列。\n\n与传统 Transformer 相比，有以下几个不同点：\n\n1.  引入视觉特征 $z$ 来进行文字序列的条件化处理，通过交叉注意力机制实现。\n2.  同时输出所有字符的概率，而不是一次只预测一个字符。（不使用自回归机制）\n3.  输出结果有两种用途：一是通过 FFN 转换为文字序列，二是输入字符感知头部来预测字符区域是否为真实字符。\n\n在训练过程中，随机抽取一个时间步 $t$，依噪声排程 $\\alpha_t, \\bar{\\alpha}_t$ 生成带有噪声的序列 $x_t$，再由 $\\text{Dec}(x_t, z, t)$ 预测 $x_{t-1}$ 并计算损失。\n\n在推理过程从填充有噪声标记（[MASK]）的序列 $x_T$ 开始，并在视觉编码器生成的视觉特征 $z$ 的条件下进行 $T$ 次迭代，最终获得 $x_0$，最后，将$x_0$经由 FFN 转换为辨识的文字序列。\n\n最后作者提出了一个字符感知头部来进行文字辨识，其主要解决固定长度序列中的字符分类问题。预测头进行二元分类，判断一个位置是否对应于一个字符（1）或非字符（0），损失函数采用二元交叉熵。\n\n## 讨论\n\n### 与其他方法的比较\n\n![方法比较](/content/papers/assets/5eac1415202bd2bf9ddc5d933a205eac49611c274c2b9fefc6c02aed9c7f6d8c.jpg)\n\n上表比较了仅用 MJ 和 ST 训练的各方法，在六个测试集上的单字准确率。\n\nDiffusionSTR 在多个数据集上表现优异，且与最新的强力方法相较具竞争力。\n\nABINet 和 PARSeq 使用了强大的语言模型，而 TRBA 则采用了图像校正模组作为前处理。尽管 DiffusionSTR 结构较简单，没有使用这些技术，但仍能达到相似的准确度。\n\n### 扩散模型的优势\n\n\n\n![识别示例](/content/papers/assets/709866602e96ac73b0b4c381aec91c9ca3f4106df811da6bc346a4ca70d21eab.jpg)\n\n\n\n从图中的辨识例子，作者观察到 DiffusionSTR 在以下情况的结果：\n\n1. **噪声图像**：部分例子比 PARSeq 辨识得更准确。\n2. **模糊图像**：部分例子仍能辨识出文字。\n\n### 字符感知头部影响\n\n![字符感知头效果](/content/papers/assets/043d25437e44bac4753fb847f6dcd3840325717dc9d52757506abffdd67598c8.jpg)\n\n实验显示了字符感知头部的效果：移除后，六个测试集的平均准确率从 91.8% 降到 63.4%。\n\n### 扩散步骤影响\n\n![扩散步数效果](/content/papers/assets/40aca308329f6eb7f28f4bcb352710bb01c8248e95560c99c6deb78f067c23b4.jpg)\n\n上表显示了总步骤数对准确度的影响：100 步到 2000 步从 17.6% 升到 91.9%，但 4000 步降为 91.3%。\n\n在 1000 步之后，这组实验的准确率大致趋于饱和。\n\n## 结论\n\n本文提出的方法是利用视觉讯息有条件地细化噪音文字序列，这和现有的方法截然不同。在验证集上，DiffusionSTR 和最先进的方法相比也毫不逊色，证明了该方法的有效性。\n\n这个新颖的思考方式，让研究者们看到了扩散模型在文字辨识上的潜力。\n\n\n> [!TIP]\n> \n> 在这篇文章中，作者只字未提推理时间。\n> \n> 由于扩散模型必须反复迭代，我们猜这个模型应该蛮慢的。\n","imageSizes":{"/content/papers/assets/fe0f3bb5164b182f59bf735f76d6487408377e22beebbfe6f11bf0f6caba5afd.jpg":{"width":1464,"height":432},"/content/papers/assets/5eac1415202bd2bf9ddc5d933a205eac49611c274c2b9fefc6c02aed9c7f6d8c.jpg":{"width":1464,"height":486},"/content/papers/assets/709866602e96ac73b0b4c381aec91c9ca3f4106df811da6bc346a4ca70d21eab.jpg":{"width":824,"height":464},"/content/papers/assets/043d25437e44bac4753fb847f6dcd3840325717dc9d52757506abffdd67598c8.jpg":{"width":1114,"height":258},"/content/papers/assets/40aca308329f6eb7f28f4bcb352710bb01c8248e95560c99c6deb78f067c23b4.jpg":{"width":1120,"height":168}}}
