{"notes":[{"id":"retail-product/rpc","category":"retail-product","legacyPath":"retail-product/1901-rpc","legacySlug":"1901-rpc","legacyPaths":[],"title":"RPC Dataset","subtitle":"老板，买单！","author":"Z. Yuan","paperDate":"2019-01-22","publishedOn":"2025-07-07","paperTitle":"RPC: A Large-Scale Retail Product Checkout Dataset","paperUrl":"https://arxiv.org/abs/1901.07249","bibliography":{"title":"RPC: A Large-Scale Retail Product Checkout Dataset","authors":["Xiu-Shen Wei","Quan Cui","Lei Yang","Peng Wang","Lingqiao Liu"],"url":"https://arxiv.org/abs/1901.07249","pdfUrl":"https://arxiv.org/pdf/1901.07249","codeUrl":null,"preprint":{"id":"1901.07249","submittedOn":"2019-01-22","url":"https://arxiv.org/abs/1901.07249"},"publication":null,"citation":"@misc{retailproductrpc2019,\n  title = {{RPC: A Large-Scale Retail Product Checkout Dataset}},\n  author = {Xiu-Shen Wei and Quan Cui and Lei Yang and Peng Wang and Lingqiao Liu},\n  year = {2019},\n  eprint = {1901.07249},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1901.07249}\n}\n","citationPath":"/content/papers/citations/retail-product/rpc.685bd79f0879.bib"},"thumbnail":{"src":"/content/papers/assets/5abf1228cb943116e0ba0fc6fbf8a2e2c2370a0aa61710dc3f38ef20c3cdfe3d.jpg","width":1084,"height":616},"locale":"zh-hans","path":"/content/papers/zh-hans/retail-product/rpc.6fc423d8a58a.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"数据集设计理念","text":"数据集设计理念","depth":2},{"id":"数据集设计","text":"数据集设计","depth":2},{"id":"数据建构流程","text":"数据建构流程","depth":2},{"id":"单品-exemplar-图像","text":"单品 exemplar 图像","depth":3},{"id":"结账图像的构建流程","text":"结账图像的构建流程","depth":3},{"id":"评估指标","text":"评估指标","depth":2},{"id":"checkout-accuracycacc","text":"Checkout Accuracy（cAcc）","depth":3},{"id":"average-counting-distanceacd","text":"Average Counting Distance（ACD）","depth":3},{"id":"mean-category-counting-distancemccd","text":"Mean Category Counting Distance（mCCD）","depth":3},{"id":"mean-category-ioumciou","text":"Mean Category IoU（mCIoU）","depth":3},{"id":"基准方法实验","text":"基准方法实验","depth":2},{"id":"single","text":"Single","depth":3},{"id":"syn","text":"Syn","depth":3},{"id":"render","text":"Render","depth":3},{"id":"synrender","text":"Syn+Render","depth":3},{"id":"详细评估表格","text":"详细评估表格","depth":3},{"id":"未来研究方向","text":"未来研究方向","depth":2},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"retail-product/dpsnet","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/deepaco","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/retail-product-recognition-review","relation":"comparison","topic":"自动结账的商品识别与计数"}]},{"id":"retail-product/dpsnet","category":"retail-product","legacyPath":"retail-product/2011-dpsnet","legacySlug":"2011-dpsnet","legacyPaths":[],"title":"DPSNet","subtitle":"结账桌上的蒸馏","author":"Z. Yuan","paperDate":"2021","publishedOn":"2025-07-06","paperTitle":"Iterative Knowledge Distillation for Automatic Check-Out","paperUrl":"https://doi.org/10.1109/tmm.2020.3037502","bibliography":{"title":"Iterative Knowledge Distillation for Automatic Check-Out","authors":["Libo Zhang","Dawei Du","Congcong Li","Yanjun Wu","Tiejian Luo"],"url":"https://doi.org/10.1109/tmm.2020.3037502","pdfUrl":"https://isrc.iscas.ac.cn/zhanglibo/pdfs/2020/IEEE_Transactions_on_Multimedia.pdf","codeUrl":null,"preprint":null,"publication":{"venue":"IEEE Transactions on Multimedia","shortVenue":"IEEE TMM","year":2021,"kind":"journal","url":"https://doi.org/10.1109/tmm.2020.3037502"},"citation":"@article{retailproductdpsnet2021,\n  title = {{Iterative Knowledge Distillation for Automatic Check-Out}},\n  author = {Libo Zhang and Dawei Du and Congcong Li and Yanjun Wu and Tiejian Luo},\n  year = {2021},\n  journal = {IEEE Transactions on Multimedia},\n  url = {https://doi.org/10.1109/tmm.2020.3037502},\n  doi = {10.1109/tmm.2020.3037502}\n}\n","citationPath":"/content/papers/citations/retail-product/dpsnet.3da4d3d4efee.bib"},"thumbnail":{"src":"/content/papers/assets/054b3451764fd6fb8748929f85e13cebee4c118717a86abdb290f1f89cbb779f.jpg","width":1556,"height":858},"locale":"zh-hans","path":"/content/papers/zh-hans/retail-product/dpsnet.c3ca3cdc4359.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"数据增强","text":"数据增强","depth":3},{"id":"dpsnet","text":"DPSNet","depth":3},{"id":"逐步知识蒸馏","text":"逐步知识蒸馏","depth":3},{"id":"损失函数","text":"损失函数","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"rpc-dataset-benchmark","text":"RPC dataset benchmark","depth":3},{"id":"消融实验","text":"消融实验","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"retail-product/rpc","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/deepaco","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/retail-product-recognition-review","relation":"comparison","topic":"自动结账的商品识别与计数"}]},{"id":"retail-product/deepaco","category":"retail-product","legacyPath":"retail-product/2206-deepaco","legacySlug":"2206-deepaco","legacyPaths":[],"title":"DeepACO","subtitle":"俯视的追猎者","author":"Z. Yuan","paperDate":"2022","publishedOn":"2025-07-20","paperTitle":"DeepACO: A Robust Deep Learning-Based Automatic Checkout System","paperUrl":"https://openaccess.thecvf.com/content/CVPR2022W/AICity/html/Pham_DeepACO_A_Robust_Deep_Learning-Based_Automatic_Checkout_System_CVPRW_2022_paper.html","bibliography":{"title":"DeepACO: A Robust Deep Learning-Based Automatic Checkout System","authors":["Long Hoang Pham","Duong Nguyen-Ngoc Tran","Huy-Hung Nguyen","Tai Huu-Phuong Tran","Hyung-Joon Jeon","Hyung-Min Jeon","Jae Wook Jeon"],"url":"https://openaccess.thecvf.com/content/CVPR2022W/AICity/html/Pham_DeepACO_A_Robust_Deep_Learning-Based_Automatic_Checkout_System_CVPRW_2022_paper.html","pdfUrl":"https://openaccess.thecvf.com/content/CVPR2022W/AICity/papers/Pham_DeepACO_A_Robust_Deep_Learning-Based_Automatic_Checkout_System_CVPRW_2022_paper.pdf","codeUrl":null,"preprint":null,"publication":{"venue":"CVPR Workshops","year":2022,"kind":"workshop","url":"https://openaccess.thecvf.com/content/CVPR2022W/AICity/html/Pham_DeepACO_A_Robust_Deep_Learning-Based_Automatic_Checkout_System_CVPRW_2022_paper.html"},"citation":"@inproceedings{retailproductdeepaco2022,\n  title = {{DeepACO: A Robust Deep Learning-Based Automatic Checkout System}},\n  author = {Long Hoang Pham and Duong Nguyen-Ngoc Tran and Huy-Hung Nguyen and Tai Huu-Phuong Tran and Hyung-Joon Jeon and Hyung-Min Jeon and Jae Wook Jeon},\n  year = {2022},\n  booktitle = {CVPR Workshops},\n  url = {https://openaccess.thecvf.com/content/CVPR2022W/AICity/html/Pham_DeepACO_A_Robust_Deep_Learning-Based_Automatic_Checkout_System_CVPRW_2022_paper.html}\n}\n","citationPath":"/content/papers/citations/retail-product/deepaco.4a3ccc35e79a.bib"},"thumbnail":{"src":"/content/papers/assets/5b6b896270577817ebce085180d5b8bf4ea36cf60ecac67ecfaee350536872ee.jpg","width":1538,"height":740},"locale":"zh-hans","path":"/content/papers/zh-hans/retail-product/deepaco.4f55e25fbd74.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"数据合成","text":"数据合成","depth":3},{"id":"模型架构","text":"模型架构","depth":3},{"id":"商品检测","text":"商品检测","depth":3},{"id":"手部估计","text":"手部估计","depth":3},{"id":"多目标跟踪","text":"多目标跟踪","depth":3},{"id":"商品计数","text":"商品计数","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"测试数据与评估指标","text":"测试数据与评估指标","depth":3},{"id":"量化成果","text":"量化成果","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"retail-product/rpc","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/dpsnet","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/retail-product-recognition-review","relation":"comparison","topic":"自动结账的商品识别与计数"}]},{"id":"retail-product/retail-product-recognition-review","category":"retail-product","legacyPath":"retail-product/2011-retail-product-recognition-review","legacySlug":"2011-retail-product-recognition-review","legacyPaths":[],"title":"RPR: Review","subtitle":"零售商品识别","author":"Z. Yuan","paperDate":"2020","publishedOn":"2025-06-30","paperTitle":"Deep Learning for Retail Product Recognition: Challenges and Techniques","paperUrl":"https://doi.org/10.1155/2020/8875910","bibliography":{"title":"Deep Learning for Retail Product Recognition: Challenges and Techniques","authors":["Yuchen Wei","Son Tran","Shuxiang Xu","Byeong Kang","Matthew Springer"],"url":"https://doi.org/10.1155/2020/8875910","pdfUrl":null,"codeUrl":null,"preprint":null,"publication":{"venue":"Computational Intelligence and Neuroscience","shortVenue":"Comput Intell Neurosci","year":2020,"kind":"journal","url":"https://doi.org/10.1155/2020/8875910"},"citation":"@article{retailproductretailproductrecognitionreview2020,\n  title = {{Deep Learning for Retail Product Recognition: Challenges and Techniques}},\n  author = {Yuchen Wei and Son Tran and Shuxiang Xu and Byeong Kang and Matthew Springer},\n  year = {2020},\n  journal = {Computational Intelligence and Neuroscience},\n  url = {https://doi.org/10.1155/2020/8875910},\n  doi = {10.1155/2020/8875910}\n}\n","citationPath":"/content/papers/citations/retail-product/retail-product-recognition-review.8911b6384e86.bib"},"thumbnail":{"src":"/content/papers/assets/d60ead687be6a48025c98f365a675cf3f998c32517488a27f7a9a8cfde6c4043.jpg","width":1508,"height":580},"locale":"zh-hans","path":"/content/papers/zh-hans/retail-product/retail-product-recognition-review.6a3aa7ff7758.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"传统方法","text":"传统方法","depth":2},{"id":"深度学习方法","text":"深度学习方法","depth":2},{"id":"商品识别的挑战","text":"商品识别的挑战","depth":2},{"id":"问题一类别规模过大","text":"问题一：类别规模过大","depth":3},{"id":"问题二数据分布落差","text":"问题二：数据分布落差","depth":3},{"id":"问题三类内变异问题","text":"问题三：类内变异问题","depth":3},{"id":"问题四系统灵活性不足","text":"问题四：系统灵活性不足","depth":3},{"id":"技术概观","text":"技术概观","depth":2},{"id":"基于卷积网络","text":"基于卷积网络","depth":3},{"id":"数据增强","text":"数据增强","depth":3},{"id":"细粒度分类","text":"细粒度分类","depth":3},{"id":"少样本学习","text":"少样本学习","depth":3},{"id":"公开数据集","text":"公开数据集","depth":2},{"id":"货架图像数据集","text":"货架图像数据集","depth":3},{"id":"结账图像数据集","text":"结账图像数据集","depth":3},{"id":"实务使用建议","text":"实务使用建议","depth":3},{"id":"未来研究方向","text":"未来研究方向","depth":2},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"retail-product/rpc","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/dpsnet","relation":"comparison","topic":"自动结账的商品识别与计数"},{"id":"retail-product/deepaco","relation":"comparison","topic":"自动结账的商品识别与计数"}]}],"text":"这篇论文做一个新的数据集，叫做 RPC（Retail Product Checkout）。\n\n我们发现很多和相关论文都会用到这个数据集。不仅如此，从这篇论文出发查找文献，能更快速地找到最新的研究成果。\n\n这么重要的节点，你肯定也不能错过。\n\n\n> [!NOTE]\n> \n> **排行榜**：[**RPC-Leaderboard**](https://github.com/RPC-Dataset/RPC-Leaderboard?tab=readme-ov-file)\n> \n> ---\n> \n> **数据集下载**：\n> \n> 1. [**https://rpc-dataset.github.io/**](https://rpc-dataset.github.io/)\n> 2. [**Kaggle - RPC Dataset**](https://www.kaggle.com/datasets/diyer22/retail-product-checkout-dataset?resource=download)\n> \n> - 我们建议使用 Kaggle 下载，比较方便。\n\n\n## 定义问题\n\n在传统零售业中，「识别商品」这件事占据了大量人力成本。随着电脑视觉技术逐渐成熟，自动结账（Automatic Check-Out, ACO）这个应用场景开始受到关注。它的目标是：**从顾客购买商品的图像中，自动生成对应的购物清单**，取代人工扫描、提升效率并降低成本。\n\n但是，这不是一个单纯的分类或检测任务，ACO 同时具备以下几个挑战特性：\n\n- **大规模（large-scale）**：商品类别数量庞大，超市中常见的 SKU 往往超过上千种；\n- **细粒度（fine-grained）**：许多商品外观极为相似，差异可能仅在小标签或包装细节；\n- **少样本（few-shot）**：大多数商品只能收集到极少的训练样本；\n- **跨域（cross-domain）**：训练数据通常为展示用的干净单品照，测试场景则来自真实结账画面，背景混乱、商品堆叠、光线不稳、遮挡严重，产生明显 domain gap。\n\n这些因素在电脑视觉中本就分别构成重大难题，而 ACO 的挑战在于：**这些问题同时存在！**\n\n过去也有一些数据集尝试处理其中部分问题，例如：\n\n- **SOIL-47** 测试光照与视角变化的影响；\n- **Supermarket Produce Dataset** 聚焦于水果与蔬菜分类；\n- **Grozi-120 / Grocery Products / Freiburg Dataset** 收录网络图与店铺实景图；\n- **MVTec D2S** 提供 pixel-level 分割标注，用于工业物体语义分割。\n\n但这些数据集通常类别数量有限，或是拍摄场景过于简化，与真实结账流程相去甚远。它们多半只解决单一子任务，无法同时测试「堆叠、多类、遮挡」等真实难题。\n\n数据量不足也让学习模型更为困难。虽然传统 data augmentation 方法（翻转、平移、色彩扰动等）能略为扩充数据分布，但无法真正模拟测试场景的噪声与语义组合。\n\n所以，最根本的问题在于：\n\n> **我们能不能建立一个足够大、足够多样化的数据集，能同时涵盖上述所有挑战？**\n\n---\n\n## 数据集设计理念\n\n\n\n![overview](/content/papers/assets/5abf1228cb943116e0ba0fc6fbf8a2e2c2370a0aa61710dc3f38ef20c3cdfe3d.jpg)\n\n\n\n在现实零售场景中，顾客将商品放上结账台时，ACO 系统的理想目标是：\n\n> **一眼就能识别出商品的种类与数量，并自动产出购物清单**。\n\n与其说这是「分类」或「检测」，不如说它更接近一种「**多物体计数与识别的组合任务**」。\n\n从定义上来看，ACO 的输入是一张拍摄于结账场景的图像，其中可能包含任意数量、任意组合的商品。而我们拥有的训练数据，却往往只有单品照。\n\n\n> [!TIP]\n> \n> **什么是单品照？**\n> \n> 单品照，原文为 **single-product exemplar image**，指的是专门拍摄单一商品的图像。这些图像通常用于电商平台或商品目录中，目的是清楚展示商品外观与细节。\n\n\n真正困难的地方在于：商品库存清单持续更新，要收集涵盖所有商品组合的结账图像几乎不可能。因此，一种更实际的做法，是利用这些单品数据进行训练，**在完全未见过商品堆叠的情况下，让模型学会在混乱中识别出多物体、多种类、多数量。**\n\n这样的问题可被形式化为：\n\n- 给定一组商品类别集合 $P = \\{p_i\\}$，与一张测试图像 $I_t$。\n- 目标即：预测所有在该图像中出现过的商品 $p$ 的出现次数 $\\text{count}(p)$。\n- 如果某商品未出现，则 $\\text{count}(p) = 0$。\n\n可使用的训练资源包含：\n\n- 一组单品图像数据集 $S = \\{(I_s, y_s)\\}$，每张图对应一项商品类别；\n- 一组可选的结账图像数据集 $C = \\{(I_c, Y_c)\\}$，其中 $Y_c$ 可能包含不同程度的标注信息。\n\n这样的设定极具挑战性，也提供了探讨「分类、计数、检测、弱监督、domain shift」等任务交集的空间。\n\n---\n\n## 数据集设计\n\n针对上述挑战，作者设计了 RPC（Retail Product Checkout）数据集，有六大关键特色：\n\n1. **大规模商品种类与样本数**\n\n   - 共收录 **200 种商品类别（SKU）**，每类约 4 件实体商品，共取得 **83,739 张图像**。\n   - 其中包含 53,739 张单品 exemplar 图，以及 30,000 张结账图像，规模远大于既有公开数据集。\n\n\n\n        ![dataset](/content/papers/assets/bcf3836f4490abafcada1f04447879c6afe4a7dd8cf3f4fb0a69ba2b2aba6545.jpg)\n\n\n\n\n\n        ![dataset infos](/content/papers/assets/176a6ae1d87360e1633ee7d39a4e0aae02bbdc51735747d3ec7cc229a8d9c58c.jpg)\n\n\n\n2. **双类型图像：单品图与堆叠图**\n\n   - 单品 exemplar 图：多角度拍摄的商品清楚图，模拟电商网页商品图。\n   - 结账图像：在模拟结账台上拍摄，包含多个商品、视角固定、堆叠与真实遮挡情况。\n\n3. **场景拟真**\n\n   - 商品在结账图中为随机挑选、随机数量、任意摆放；\n   - 含有大量遮挡、旋转、多物体重叠，模拟实际零售流程的复杂性。\n\n4. **层级式商品分类结构**\n\n   - 所有商品共分为 **17 个上位类别（meta-categories）**，如饮料、零食、文具等；瓶状、盒状、袋状是其外观形态。\n   - 有助于模型学习类别间语义结构，亦可作为辅助监督讯号，或阶层分类的研究方向。\n\n5. **杂乱程度标注**\n\n\n\n    ![clutter levels](/content/papers/assets/950c5984aa2302b080f8756ec5e1f98b99730ef2c0cc74af2b522d929936967b.jpg)\n\n\n\n   - 根据每张结账图中的商品种类与数量，将其划分为三种 clutter level，有助于分析模型在不同复杂度场景下的表现稳定性。\n\n6. **三种监督强度**\n\n\n\n    ![week to strong](/content/papers/assets/8c97a487cebb1d173c1895e9469f8152e59cb4d1c3ce72c9cc39185b4a725198.jpg)\n\n\n\n   - **弱监督**：购物清单（仅提供商品种类与数量）；\n   - **中监督**：点位标注（标示每个商品的中心点与类别）；\n   - **强监督**：边界框（完整标示每个商品的 bounding box 与类别）。\n\n这些设计让 RPC 成为一个高度可扩充的 benchmark，不仅能评估模型识别与计数能力，更能延伸至弱监督检测、domain adaptation、多尺度分类等多种研究路径。\n\n## 数据建构流程\n\n![meta-categories](/content/papers/assets/c3fb023608ede295886d884c5eb9aceaead530363f5d3293eff9970b756e9f9a.jpg)\n\n本数据集共收录 **200 个零售商品类别（SKU）**，依照外观与功能分为 **17 个上位类别（meta-categories）**，涵盖从食品到生活用品的多元型态：\n\n> puffed food、dried fruit、dried food、instant drink、instant noodles、dessert、drink、alcohol、milk、canned food、chocolate、gum、candy、seasoner、personal hygiene、tissue、stationery。\n\n这些类别对应的外观结构多样，从瓶状、盒状、罐状到袋状都有涵盖。\n\n同一 meta-category 下的 SKU 通常具有高度外观相似性，例如同样是「果汁」，可能只差一条文字或标签颜色，这正是 ACO 任务中最棘手的「细粒度识别」难题之一。\n\n---\n\n### 单品 exemplar 图像\n\n\n\n![dataset infos](/content/papers/assets/0634e7adb176ea790ae7e32ee0d790232d096f4e39f5b0fd2c3596c356367b5e.jpg)\n\n\n\n为了获得每个商品在多视角下的外观信息，作者设计了一套自动化拍摄流程：\n\n- 使用 **四台固定视角摄像机**，分别对应：上视角（top view）、水平视角（side view）、30° 与 45° 斜视角；\n- 每个商品被放置于可旋转的 **360 度转盘** 上，每 9 度拍摄一次，共取得 40 张图像；\n- 每台摄像机各拍摄 40 张 → 每个 SKU 共产生 **160 张视角图像**；\n- 若商品具有上下外观差异（如盒状、袋状），则会进行「翻面后再重拍一次」，以补齐底部特征。\n\n最终，这一阶段共拍摄 **53,739 张 exemplar 图像**，全部为**干净背景、多视角、单一商品**的样貌，对应于训练阶段常见的 source domain 图像。\n\n\n\n![dataset infos](/content/papers/assets/29e462eeb3314dfe697b2b5a392e70e554a44cb8c2ee488157959e0a05c65f35.jpg)\n\n\n\n---\n\n### 结账图像的构建流程\n\n\n\n![dataset infos](/content/papers/assets/0c75f04034d9ef1d5d80e75690c634c87ba49bc8760b21e19ca2c007b9e5648a.jpg)\n\n\n\n与 exemplar 不同，结账图像的目标是模拟真实结账情境，因此设计上刻意导入混乱与遮挡：\n\n- 拍摄背景为一张 **80cm × 80cm 白板**，摄像机从正上方拍摄，分辨率为 1800 × 1800；\n- 每张图中放置的商品种类与数量均为随机组合，但依据预设的「杂乱程度」进行控制；\n- 商品会随机摆放、角度任意、不进行人工对齐，进一步模拟多件商品堆叠的真实情境。\n\n根据遮挡与密度程度，数据集将结账图像分为三种杂乱等级：\n\n\n\n| 杂乱等级 | 商品种类数         | 商品实例总数 | 特征                     |\n| -------- | ------------------ | ------------ | ------------------------ |\n| Easy     | 3–5 种           | 3–10 件      | 数量较少                 |\n| Medium   | 5–8 种           | 10–15 件     | 数量居中                 |\n| Hard     | 8–10 种          | 15–20 件     | 数量较多，遮挡更具挑战   |\n\n\n\n每个杂乱等级都各收录 10,000 张图像，总计构成 **30,000 张结账图像**。\n\n## 评估指标\n\n自动结账任务的目标是：**正确预测一张图像中所有商品的种类与数量**。\n\n为了量化模型在这类任务上的表现，RPC 数据集提出了一套针对 ACO 任务设计的专属评估指标，从单张图像准确性、数量误差、类别层次表现到语义一致性皆有考量。\n\n以下是主要的四个核心指标。\n\n### Checkout Accuracy（cAcc）\n\ncAcc 代表整体准确率。\n\nACO 系统的最终目的是在**一张图像中，完整预测出购物清单**，因此最核心的指标是：\n\n> **若模型预测的所有商品种类与数量皆与 GT 完全一致，则该张图像判定为「正确」。**\n\n这项指标的公式如下：\n\n$$\ncAcc = \\frac{1}{N} \\sum_{i=1}^{N} \\delta \\left( \\sum_{k=1}^{K} |P_{i,k} - GT_{i,k}|, 0 \\right)\n$$\n\n- $P_{i,k}$：模型对第 $i$ 张图像中第 $k$ 类商品的预测数量；\n- $GT_{i,k}$：该类商品的真实数量；\n- $\\delta(a,b)$：若两个输入相等则回传 1，否则为 0；此处比较的是总误差与 0。\n\n这是一个极为严格的指标，只要预测错一件商品，就会判定该张图像不通过。\n\n### Average Counting Distance（ACD）\n\nACD 表示平均数量误差。\n\n若不要求「完全正确」，而是评估模型平均犯错几次，可采用 ACD：\n\n> **衡量每张图像中，模型对所有类别的总体预测误差（L1 距离）。**\n\n公式如下：\n\n$$\nACD = \\frac{1}{N} \\sum_{i=1}^{N} \\sum_{k=1}^{K} |P_{i,k} - GT_{i,k}|\n$$\n\n这项指标适合用于比较模型在各类商品数量预测上的误差；若把一件商品错认成另一类，两个类别的计数都可能产生误差。\n\n---\n\n### Mean Category Counting Distance（mCCD）\n\nmCCD 表示类别平均误差比率，其将注意力放在各商品类别的识别难度上：\n\n> **衡量模型在每个类别上，预测误差与实际数量的比例，平均所有类别。**\n\n公式如下：\n\n$$\nmCCD = \\frac{1}{K} \\sum_{k=1}^{K} \\frac{ \\sum_{i=1}^{N} |P_{i,k} - GT_{i,k}| }{ \\sum_{i=1}^{N} GT_{i,k} }\n$$\n\n这项指标反映了模型是否在某些类别（如细粒度商品）上容易高估或低估，可用来评估类别不平衡问题与长尾分布下的预测稳定性。\n\n---\n\n### Mean Category IoU（mCIoU）\n\nmCIoU 表示购物清单相似度。\n\n最后，作者定义了一个类似 IoU（Intersection over Union）的指标，用来衡量预测与实际购物清单在语义层次上的接近程度：\n\n$$\nmCIoU = \\frac{1}{K} \\sum_{k=1}^{K} \\frac{ \\sum_{i=1}^{N} \\min(P_{i,k}, GT_{i,k}) }{ \\sum_{i=1}^{N} \\max(P_{i,k}, GT_{i,k}) }\n$$\n\n- 若某类商品预测数量与实际数量越接近，该项在分子与分母中比值越高；\n- 对某类商品而言，若在整个评估集的预测或标注中只有一边出现，该类别的比值为 0。\n\nmCIoU 是一项**按类别比较预测与真值计数重叠程度**的指标，能分析「即便购物清单未完全正确，模型预测的类别与数量有多接近」。\n\n## 基准方法实验\n\n由于自动结账问题本身是跨域、少样本、细粒度的复合型任务，目前尚无统一的解法架构。\n\n为建立 RPC 数据集上的 benchmark，作者尝试将 ACO 问题视为一种 **cross-domain detection task**，并提出四种逐步递进的基线方法。检测器的监督标注只取自 exemplar 单品图，不使用结账图的标注；Render 方法另以结账图的外观作为风格转换目标。\n\n---\n\n### Single\n\n这个方法是直接用 exemplar 训练 FPN。\n\n最直觉的策略，是直接将单品 exemplar 图作为监督数据，训练一个物体检测模型。作者选用 **Feature Pyramid Network (FPN)** 作为检测器。\n\n然而这个 baseline 几乎无法泛化到结账图，因为 exemplar 是单物体、无遮挡的干净图，而测试数据却是多物体、堆叠遮挡、光影混乱的 domain。这个 domain gap 使得模型即使在 easy 模式下也表现极差，成为最弱的 baseline。\n\n---\n\n### Syn\n\n这个策略是合成结账场景图进行训练。\n\n为了缩小这个 domain gap，第二种方法将 exemplar 商品图「剪下贴上」，**合成出模拟结账的图像**。具体步骤如下：\n\n- 使用显著区域分割与 CRF（条件随机场）后处理取得商品 mask；\n- 将多个商品随机贴到空白背景上，建立多物体堆叠场景；\n- 依据不同 clutter 等级规则，合成出 **10 万张**训练用的合成结账图。\n\n这种策略让训练数据更接近测试场景，cAcc 提升明显，尤其在 easy clutter 模式下从 0.02% 升至 18.49%，差 **18.47 个百分点**；但在 medium 与 hard 模式分别只有 6.54% 与 2.91%，纯合成图仍不足以填补此处的 domain gap。\n\n---\n\n### Render\n\n\n\n![cycle-gan](/content/papers/assets/77a5560dc526b6bacc36763117787034e211fd62e72c122c387151591519681a.jpg)\n\n\n\n这个方法采用 Cycle-GAN 跨域风格转换。\n\n如上图所示，左边是简单拼贴的合成结账图，右边是经过 Cycle-GAN 风格转换后的图像，看起来更自然、更接近真实结账场景，整体流程为：\n\n- 对合成图进行 Cycle-GAN 翻译，使其风格更接近真实结账图；\n- 使用翻译后图像训练 FPN 检测器。\n\n这种 **domain translation** 策略在三种 clutter 模式下的 cAcc 都高于 Syn，显示转换后的图像在这组跨域检测实验中有帮助。\n\n---\n\n### Syn+Render\n\n最后一种策略是混合数据训练，提升样本多样性。\n\n作者尝试将 **10 万张 render 后的数据**与**10 万张原始 synthetic 合成图**混合训练，期望通过样本多样性进一步提升模型泛化能力，结果显示比单独使用 Syn 或 Render 效果更佳。\n\n这说明数据风格的一致性（render）与样本组合的多样性（syn）两者可互补，有助于模型学习更稳健的判断边界。\n\n---\n\n### 详细评估表格\n\n\n\n![benchmark](/content/papers/assets/846253dfb5082a2ea6d69c1d9ed1d884f2dced6a89ab3131177594638fd5a60c.jpg)\n\n\n\n实验同时采用：\n\n- ACO 任务指标（cAcc、ACD、mCCD、mCIoU）；\n- 标准检测指标（mAP\\@50, mmAP）。\n\n训练设定如下：\n\n- 输入图像缩放最短边为 800px；\n- 使用同步 SGD，两张图/卡 × 2 GPU；\n- 每张图最多采样 512 个 RoI；\n- 起始学习率 0.02，60K iter 后降为 0.002；\n- 各 clutter 模式中，使用 2,000 张图作验证集，8,000 张作测试集。\n\n实验结果显示：最佳的「Syn+Render」在 easy、medium、hard 模式的 cAcc 分别为 73.17%、54.69%、42.48%；后两者仍有大量完整购物清单预测错误。论文的失败案例分析也指出堆叠遮挡与细粒度识别等困难。\n\n尽管在 mAP 指标下表现良好（如 hard 模式下达到 **72.72% mmAP**），但从 cAcc 角度看仍远远不足，**正确识别整张图的完整购物清单**仍是一项极严格的任务要求。\n\n失败案例分析指出主要错误来源包含：\n\n1. 漏检（missed detection）；\n2. 密集排列造成遮挡；\n3. 细粒度商品误辨；\n4. 误判（false positives）。\n\n这些挑战也为后续研究指出明确方向：**弱监督、多视角融合、语义对齐、样本生成多样性**皆是未来可挖掘的切入点。\n\n\n> [!TIP]\n> \n> 这篇论文提到所使用的架构是 FPN，但没有说明是用 ResNet-50 还是 ResNet-101。\n> \n> DPNet 论文中的对照实验曾使用 ResNet-101，但不能据此推定本篇 RPC 基准方法的 FPN 骨干也相同。\n> \n> 参考文献如下：[**[19.04] Data Priming Network for Automatic Check-Out**](https://arxiv.org/abs/1904.04978)\n\n\n## 未来研究方向\n\n虽然本文以 cross-domain detection 为切入点建立了 RPC 的 baseline，但 ACO 任务的本质远比单一解法更为开放与多元。RPC 数据集提供了可弹性扩展的结构与标注方式，因此也启发了许多值得探索的潜在研究路线：\n\n- **线上学习与动态扩充**：\n\n  现实情境中，商品清单并非固定不变，新品持续上架意味着系统需要**具备即时学习与增量更新的能力**。如何不重训整个模型，快速将新商品纳入识别范围，构成一个典型的 online learning 问题。特别是 ACO 的跨域与细粒度特性让这项任务更加困难，可能需要全新的架构与策略设计。\n\n- **跳过检测的清单预测模型**：\n\n  一种不同的方向，是**不通过 bounding box 或检测流程**，而直接从结账图像预测购物清单，将问题转化为一种多类别多实例的**物体计数（object counting）任务**。\n\n  但与传统 counting 任务不同，ACO 涉及数百种物体、极少样本、外观高度相似，因此可视为一种全新形式的跨类别计数问题。\n\n- **混合监督的学习架构**：\n\n  RPC 提供了三种监督强度的 checkout 标注（list-level, point-level, bbox-level），但如何有效结合这些异质标注资源，设计一个**具适应性与抽象能力的混合监督学习框架**，仍是一个尚未解决的开放问题，尤其在数据收集成本极高的场域下格外重要。\n\n- **延伸至其他电脑视觉任务的应用场景**：\n\n  尽管 RPC 是为自动结账而生，其图像与标注架构也可支持其他电脑视觉研究方向，如：\n\n  - **物体检索（object retrieval）**\n  - **少样本学习（few-shot detection）**\n  - **弱监督学习（weakly-supervised detection）**\n  - **全监督检测（fully-supervised detection）**\n\n  特别是结账图中提供了完整 bounding box 与 SKU 类别标注，使其具备训练与评估检测模型的条件，可作为通用物体检测任务的补充数据集之一。\n\n## 结论\n\n本篇论文发表于 2019 年，而我们回顾它的时刻已经是 2025 年。\n\n在这几年间，大型视觉语言模型快速发展，对图像中商品的识别与推理能力已有初步展现，不少人开始探索其应用于自动结账场景的可能性。\n\n不过目前似乎还没有看到可行的进展，原因可能在于 ACO 问题中的细粒度识别、遮挡处理与跨域稳定性到问题依然没办法被完全解决。\n\n这也显示出 RPC 数据集的价值：它不仅是过去的里程碑，更是未来研究的基石。希望未来能有更多研究者在此基础上，提出创新的方法与解决方案，推动自动结账技术的实用化。\n","imageSizes":{"/content/papers/assets/5abf1228cb943116e0ba0fc6fbf8a2e2c2370a0aa61710dc3f38ef20c3cdfe3d.jpg":{"width":1084,"height":616},"/content/papers/assets/bcf3836f4490abafcada1f04447879c6afe4a7dd8cf3f4fb0a69ba2b2aba6545.jpg":{"width":1088,"height":748},"/content/papers/assets/176a6ae1d87360e1633ee7d39a4e0aae02bbdc51735747d3ec7cc229a8d9c58c.jpg":{"width":1100,"height":476},"/content/papers/assets/950c5984aa2302b080f8756ec5e1f98b99730ef2c0cc74af2b522d929936967b.jpg":{"width":816,"height":224},"/content/papers/assets/8c97a487cebb1d173c1895e9469f8152e59cb4d1c3ce72c9cc39185b4a725198.jpg":{"width":1104,"height":428},"/content/papers/assets/c3fb023608ede295886d884c5eb9aceaead530363f5d3293eff9970b756e9f9a.jpg":{"width":1514,"height":378},"/content/papers/assets/0634e7adb176ea790ae7e32ee0d790232d096f4e39f5b0fd2c3596c356367b5e.jpg":{"width":1096,"height":712},"/content/papers/assets/29e462eeb3314dfe697b2b5a392e70e554a44cb8c2ee488157959e0a05c65f35.jpg":{"width":1076,"height":520},"/content/papers/assets/0c75f04034d9ef1d5d80e75690c634c87ba49bc8760b21e19ca2c007b9e5648a.jpg":{"width":868,"height":1080},"/content/papers/assets/77a5560dc526b6bacc36763117787034e211fd62e72c122c387151591519681a.jpg":{"width":740,"height":1080},"/content/papers/assets/846253dfb5082a2ea6d69c1d9ed1d884f2dced6a89ab3131177594638fd5a60c.jpg":{"width":1526,"height":702}}}
