{"notes":[{"id":"lightweight/fasternet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FasterNet","subtitle":"FLOPs 会骗人","author":"Z. Yuan","paperDate":"2023-03-07","publishedOn":"2025-10-23","world":"external/fasternet","paperTitle":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","paperUrl":"https://arxiv.org/abs/2303.03667","bibliography":{"title":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","authors":["Jierun Chen","Shiu-hong Kao","Hao He","Weipeng Zhuo","Song Wen","Chul-Ho Lee","S.-H. Gary Chan"],"url":"https://arxiv.org/abs/2303.03667","pdfUrl":"https://arxiv.org/pdf/2303.03667v3","codeUrl":null,"preprint":{"id":"2303.03667","submittedOn":"2023-03-07","url":"https://arxiv.org/abs/2303.03667"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Chen_Run_Dont_Walk_Chasing_Higher_FLOPS_for_Faster_Neural_Networks_CVPR_2023_paper.html"},"citation":"@misc{lightweightfasternet2023,\n  title = {{Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks}},\n  author = {Jierun Chen and Shiu-hong Kao and Hao He and Weipeng Zhuo and Song Wen and Chul-Ho Lee and S.-H. Gary Chan},\n  year = {2023},\n  eprint = {2303.03667},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2303.03667}\n}\n","citationPath":"/content/papers/citations/lightweight/fasternet.7a313d25fdbe.bib"},"thumbnail":{"src":"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png","width":1012,"height":500},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/fasternet.46f8910dbb1b.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"延迟等于-flops-除以-flops","text":"延迟等于 FLOPs 除以 FLOPS","depth":3},{"id":"depthwise-卷积慢在哪里","text":"depthwise 卷积慢在哪里","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"部分卷积-pconv","text":"部分卷积 PConv","depth":3},{"id":"再接一个-11-卷积","text":"再接一个 1×1 卷积","depth":3},{"id":"fasternet","text":"FasterNet","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"检测与分割","text":"检测与分割","depth":3},{"id":"消融","text":"消融","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/shufflenet","relation":"referenced","topic":""},{"id":"lightweight/ghostnet","relation":"referenced","topic":""}]},{"id":"lightweight/mobilenet-v2","category":"lightweight","legacyPath":"lightweight/1801-mobilenet-v2","legacySlug":"1801-mobilenet-v2","legacyPaths":[],"title":"MobileNet-V2","subtitle":"瓶颈内的淬炼","author":"Z. Yuan","paperDate":"2018-01-13","publishedOn":"2024-06-27","paperTitle":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","paperUrl":"https://arxiv.org/abs/1801.04381","bibliography":{"title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","authors":["Mark Sandler","Andrew Howard","Menglong Zhu","Andrey Zhmoginov","Liang-Chieh Chen"],"url":"https://arxiv.org/abs/1801.04381","pdfUrl":"https://arxiv.org/pdf/1801.04381","codeUrl":null,"preprint":{"id":"1801.04381","submittedOn":"2018-01-13","url":"https://arxiv.org/abs/1801.04381"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://arxiv.org/abs/1801.04381"},"citation":"@misc{lightweightmobilenetv22018,\n  title = {{MobileNetV2: Inverted Residuals and Linear Bottlenecks}},\n  author = {Mark Sandler and Andrew Howard and Menglong Zhu and Andrey Zhmoginov and Liang-Chieh Chen},\n  year = {2018},\n  eprint = {1801.04381},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.04381}\n}\n","citationPath":"/content/papers/citations/lightweight/mobilenet-v2.a882707cbaa2.bib"},"thumbnail":{"src":"/content/papers/assets/e6bbfb346bb449f83897d76b69c3711d2e9937d40320a7aab0f3843653a688c0.jpg","width":2332,"height":1324},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/mobilenet-v2.3cc14e69626b.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"线性瓶颈","text":"线性瓶颈","depth":3},{"id":"残差网络","text":"残差网络","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"反转残差","text":"反转残差","depth":3},{"id":"扩张瓶颈论文附录-a","text":"扩张瓶颈（论文附录 A）","depth":3},{"id":"模型架构","text":"模型架构","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"图像分类性能","text":"图像分类性能","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v1","relation":"background","topic":"移动设备上的高效卷积"},{"id":"lightweight/mobilenet-v3","relation":"next","topic":"移动设备上的高效卷积"}]},{"id":"lightweight/shufflenet","category":"lightweight","legacyPath":"lightweight/1707-shufflenet","legacySlug":"1707-shufflenet","legacyPaths":[],"title":"ShuffleNet","subtitle":"通道洗牌网络","author":"Z. Yuan","paperDate":"2017-07-04","publishedOn":"2024-06-26","paperTitle":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","paperUrl":"https://arxiv.org/abs/1707.01083","bibliography":{"title":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","authors":["Xiangyu Zhang","Xinyu Zhou","Mengxiao Lin","Jian Sun"],"url":"https://arxiv.org/abs/1707.01083","pdfUrl":"https://arxiv.org/pdf/1707.01083","codeUrl":null,"preprint":{"id":"1707.01083","submittedOn":"2017-07-04","url":"https://arxiv.org/abs/1707.01083"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://doi.org/10.1109/cvpr.2018.00716"},"citation":"@misc{lightweightshufflenet2017,\n  title = {{ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices}},\n  author = {Xiangyu Zhang and Xinyu Zhou and Mengxiao Lin and Jian Sun},\n  year = {2017},\n  eprint = {1707.01083},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1707.01083}\n}\n","citationPath":"/content/papers/citations/lightweight/shufflenet.1e5a6869f5a9.bib"},"thumbnail":{"src":"/content/papers/assets/6d0c746d403d2823809ffbd52c9b3614cc2063afee07608d6d0cb26b11fc3ac8.jpg","width":1746,"height":678},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/shufflenet.10b18ea282ec.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"计算量与速度不一致","text":"计算量与速度不一致","depth":3},{"id":"大量-1x1-卷积","text":"大量 1x1 卷积","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"组卷积","text":"组卷积","depth":3},{"id":"通道洗牌","text":"通道洗牌","depth":3},{"id":"洗牌模块放置位置","text":"洗牌模块放置位置","depth":3},{"id":"模型架构","text":"模型架构","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"组卷积的效果","text":"组卷积的效果","depth":3},{"id":"通道洗牌的效果","text":"通道洗牌的效果","depth":3},{"id":"与其他模型的比较","text":"与其他模型的比较","depth":3},{"id":"真实推理速度评估","text":"真实推理速度评估","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/ghostnet","relation":"comparison","topic":"降低卷积与通道的计算成本"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"降低卷积与通道的计算成本"}]},{"id":"lightweight/ghostnet","category":"lightweight","legacyPath":"lightweight/1911-ghostnet","legacySlug":"1911-ghostnet","legacyPaths":[],"title":"GhostNet","subtitle":"特征图中的幽灵","author":"Z. Yuan","paperDate":"2019-11-27","publishedOn":"2024-07-23","paperTitle":"GhostNet: More Features from Cheap Operations","paperUrl":"https://arxiv.org/abs/1911.11907","bibliography":{"title":"GhostNet: More Features from Cheap Operations","authors":["Kai Han","Yunhe Wang","Qi Tian","Jianyuan Guo","Chunjing Xu","Chang Xu"],"url":"https://arxiv.org/abs/1911.11907","pdfUrl":"https://arxiv.org/pdf/1911.11907","codeUrl":"https://github.com/huawei-noah/ghostnet","preprint":{"id":"1911.11907","submittedOn":"2019-11-27","url":"https://arxiv.org/abs/1911.11907"},"publication":{"venue":"CVPR","year":2020,"kind":"conference","url":"https://arxiv.org/abs/1911.11907"},"citation":"@misc{lightweightghostnet2019,\n  title = {{GhostNet: More Features from Cheap Operations}},\n  author = {Kai Han and Yunhe Wang and Qi Tian and Jianyuan Guo and Chunjing Xu and Chang Xu},\n  year = {2019},\n  eprint = {1911.11907},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1911.11907}\n}\n","citationPath":"/content/papers/citations/lightweight/ghostnet.4dd65d9540bb.bib"},"thumbnail":{"src":"/content/papers/assets/22fa9d36b042deb7995e9695a803de3a973542f58cd9cd48e37ef13167df2222.jpg","width":972,"height":904},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/ghostnet.e84a905fbb3f.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"模块配置","text":"模块配置","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet-上的性能","text":"ImageNet 上的性能","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/shufflenet","relation":"comparison","topic":"降低卷积与通道的计算成本"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"降低卷积与通道的计算成本"}]}],"text":"设计轻量网络的人，最常看的数字是 FLOPs。MobileNet、ShuffleNet、GhostNet 一路下来，大家想尽办法把浮点运算量压低，最常用的工具是 depthwise 卷积和分组卷积。\n\n\n> [!TIP]\n> \n> 这几篇可以先看我们的笔记：\n> \n> - **[[18.01] MobileNet-V2：瓶颈内的淬炼](/blog/papers/lightweight/mobilenet-v2)**\n> - **[[17.07] ShuffleNet：通道洗牌网络](/blog/papers/lightweight/shufflenet)**\n> - **[[19.11] GhostNet：特征图中的幽灵](/blog/papers/lightweight/ghostnet)**\n\n\n可是 FLOPs 少了，网络跑得就一定快吗？香港科技大学的 Jierun Chen 等人测了一圈，发现答案常常是否定的。例如 CycleMLP-B1 的 FLOPs 只有 ResNet-50 的一半，在 CPU 上却比较慢，116.1 ms 对 73.0 ms。这篇的标题叫「Run, Don't Walk」，意思是与其省步数，不如跑快一点。\n\n## 定义问题\n\n### 延迟等于 FLOPs 除以 FLOPS\n\n这两个词只差一个大小写。FLOPs 是浮点运算的次数，FLOPS 是每秒能做多少次浮点运算，也就是硬件实际跑出来的速度。延迟是两者相除：\n\n$$\n\\text{Latency}=\\frac{\\text{FLOPs}}{\\text{FLOPS}} .\n$$\n\n只压低 FLOPs，如果 FLOPS 掉得更多，延迟反而会变长。作者在 Intel CPU 上测了一批常见的网络：\n\n[![两张图。(a) 横轴是 FLOPs（G），纵轴是 CPU 上实际的 FLOPS（G/s）。橙色星号的 FasterNet 在最上方，从约 37 一路升到 64；ResNet50 是一个约 56 的单点；PVT、PoolFormer、MobileViT、EdgeNeXt 大约落在 13 到 43 之间；MobileNetV2、GhostNet、ShuffleNetV2、MobileFormer、CycleMLP 多数在 30 以下，MobileFormer 和 GhostNet 最低，只有个位数到十几。(b) 横轴是 FLOPs，纵轴是 CPU 延迟（ms），右边是完整范围，左边放大 1.8 GFLOPs 以下的部分；同样的 FLOPs 下，FasterNet 的橙线都在最下方，CycleMLP 的两个点在最上方，约 116 和 186 ms](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)\n\n*原论文 Figure 2，取自 [**v3 PDF 第 2 页**](https://arxiv.org/pdf/2303.03667v3#page=2)，CC BY 4.0。(a) CPU 上的 FLOPS 对 FLOPs；(b) CPU 延迟对 FLOPs。*\n\n左图的纵轴是每秒实际做了多少运算。许多号称快速的网络，FLOPS 都比 ResNet-50 低，硬件的算力没有用上。\n\n这个现象 ShuffleNet V2 在 2018 年就提过，它也指出 FLOPs 和速度对不上。可是之后的轻量网络还是继续用 depthwise 卷积，因为一直没有更好的替代品。\n\n### depthwise 卷积慢在哪里\n\n输入是 $c\\times h\\times w$ 的特征图，3×3 卷积核的边长 $k=3$。一般卷积的 FLOPs 是 $h\\times w\\times k^2\\times c^2$，depthwise 卷积每个通道只用自己的卷积核，FLOPs 降到 $h\\times w\\times k^2\\times c$，少了 $c$ 倍。\n\n问题在于 depthwise 卷积只做空间方向的运算，单独用的话准确率掉很多，所以实际上都会把通道数放大，MobileNet-V2 的倒残差块就放大了 6 倍。通道从 $c$ 变成 $c'$ 之后，内存访问量是\n\n$$\nh\\times w\\times 2c' + k^2\\times c' \\approx h\\times w\\times 2c' ,\n$$\n\n读一次输入、写一次输出，几乎全是特征图的搬运。一般卷积的访存量约是 $h\\times w\\times 2c$，反而比较少。\n\n运算少、搬运多，每搬一个数只做几次运算，硬件就会一直在等内存。\n\n## 解决问题\n\n### 部分卷积 PConv\n\n作者观察到，特征图的不同通道之间常常很像。他们把预训练 ResNet-50 中间层的特征图画出来，很多通道看起来非常相似。既然这么冗余，空间方向的卷积就不必每个通道都做。\n\n[![三种卷积的示意图。(a) 一般卷积：一个长方体输入和一整排卷积核做卷积，得到同样大小的输出。(b) depthwise 或分组卷积：输入沿通道方向被虚线切成很多片，每片各自配一个小卷积核。(c) 部分卷积：输入只有前面一小段通道用虚线框出来，配上少数几个卷积核做卷积，剩下的通道沿着一条箭头原封不动地接到输出](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)\n\n*原论文 Figure 1，取自 [**v3 PDF 第 1 页**](https://arxiv.org/pdf/2303.03667v3#page=1)，CC BY 4.0。*\n\n部分卷积（Partial Convolution，PConv）只对前 $c_p$ 个通道做一般的 3×3 卷积，剩下的通道直接复制到输出。取前面连续的一段，是为了让内存访问保持连续。默认 $c_p = c/4$，FLOPs 是\n\n$$\nh\\times w\\times k^2\\times c_p^2 ,\n$$\n\n只有一般卷积的 1/16；内存访问量约为 $h\\times w\\times 2c_p$，是一般卷积的 1/4。\n\n我们在自己的 CPU 上测测看。三种算子各叠 10 层，输入是 192×28×28，另外补一组把通道放大 6 倍的 depthwise 卷积，模拟 MobileNet-V2 的用法：\n\n```python\nimport time\nimport torch\nimport torch.nn as nn\n\ntorch.set_num_threads(1)                                   # 和论文一样，CPU 只用单线程\nc, h, w = 192, 28, 28                                      # 第二阶段常见的特征图大小\ncp = c // 4                                                # PConv 只对 1/4 的通道做卷积\n\nclass PConv(nn.Module):\n    def __init__(self, c, cp):\n        super().__init__()\n        self.cp = cp\n        self.conv = nn.Conv2d(cp, cp, 3, padding=1, bias=False)\n\n    def forward(self, x):\n        x1, x2 = torch.split(x, [self.cp, x.shape[1] - self.cp], dim=1)\n        return torch.cat((self.conv(x1), x2), 1)           # 其余 3/4 的通道原封不动\n\n# (算子, 输入通道数, FLOPs, 内存访问量)；FLOPs 按乘加次数计，访存量是读写特征图加上读权重的元素数\nops = {\n    \"Conv 3×3\": (nn.Conv2d(c, c, 3, padding=1, bias=False), c, h * w * 9 * c * c, h * w * 2 * c + 9 * c * c),\n    \"DWConv 3×3\": (nn.Conv2d(c, c, 3, padding=1, groups=c, bias=False), c, h * w * 9 * c, h * w * 2 * c + 9 * c),\n    \"PConv 3×3\": (PConv(c, cp), c, h * w * 9 * cp * cp, h * w * 2 * cp + 9 * cp * cp),\n    \"DWConv ×6\": (nn.Conv2d(6 * c, 6 * c, 3, padding=1, groups=6 * c, bias=False), 6 * c,\n                  h * w * 9 * 6 * c, h * w * 2 * 6 * c + 9 * 6 * c),\n}\nwith torch.no_grad():\n    for name, (op, cin, flops, mem) in ops.items():\n        x = torch.randn(1, cin, h, w)\n        net = nn.Sequential(*[op] * 10)                    # 同一层叠 10 次\n        for _ in range(20):\n            net(x)                                         # 预热\n        t = time.perf_counter()\n        for _ in range(200):\n            net(x)\n        ms = (time.perf_counter() - t) / 200 * 1000\n        total = flops * 10\n        print(f\"{name:<10}  FLOPs {total / 1e6:6.1f}M  每访问一个数做 {flops / mem:5.1f} 次乘加  \"\n              f\"延迟 {ms:5.2f} ms  FLOPS {total / ms / 1e6:4.1f} G/s\")\n```\n\n```\nConv 3×3    FLOPs 2601.1M  每访问一个数做 411.0 次乘加  延迟 32.83 ms  FLOPS 79.2 G/s\nDWConv 3×3  FLOPs   13.5M  每访问一个数做   4.5 次乘加  延迟  0.83 ms  FLOPS 16.4 G/s\nPConv 3×3   FLOPs  162.6M  每访问一个数做 169.3 次乘加  延迟  2.72 ms  FLOPS 59.8 G/s\nDWConv ×6   FLOPs   81.3M  每访问一个数做   4.5 次乘加  延迟 10.29 ms  FLOPS  7.9 G/s\n```\n\n这是在 Intel i9-13900K 上跑的结果，换一台机器数字会不同，趋势应该差不多。\n\n先看 FLOPS 那一列。一般卷积每访问一个数要做 411 次乘加，硬件用得最满；depthwise 卷积只做 4.5 次，FLOPS 掉到五分之一左右。PConv 每个数做 169 次，FLOPS 大约是一般卷积的四分之三。\n\n公平地说，同样 192 个通道、同样叠 10 层，不加宽的 depthwise 卷积其实最快，只要 0.83 ms，因为它的 FLOPs 实在太少了。可是它不会这样单独用。通道放大 6 倍之后，FLOPs 只有 PConv 的一半，延迟却是 PConv 的将近 4 倍。\n\n论文用 2080Ti、i9-9900X 和 Cortex-A72 做了同样的实验，四种特征图大小平均起来，PConv 的 FLOPs 是一般卷积的 1/16，FLOPS 则是 depthwise 卷积的 10.5 倍（GPU）、6.2 倍（CPU）和 22.8 倍（ARM）。\n\n### 再接一个 1×1 卷积\n\nPConv 只看了四分之一的通道，剩下的信息要靠后面的 1×1 卷积补上。两者合起来的感受野像一个 T 字，中心那个位置看了所有通道，周围的八个位置只看了 $c_p$ 个通道。\n\n作者用预训练的 ResNet-18 检查这个形状合不合理。对每个 3×3 卷积核，算九个位置各自的权重大小，看哪个位置最大；四个阶段里，最常成为最大的都是正中间那个位置。中心本来就比周围重要，多花一点运算在中心上是合理的。\n\n那为什么不直接用一个 T 字形的卷积？因为拆开比较省。T 字形卷积的 FLOPs 是 $h\\times w\\times(k^2 c_p c + c(c-c_p))$，拆成 PConv 加 1×1 卷积是 $h\\times w\\times(k^2 c_p^2 + c^2)$。$c_p = c/4$ 时，前者是 $3c^2$，后者只有 $1.5625c^2$。\n\n作者也验证了这个组合能不能取代一般卷积。从预训练 ResNet-50 每个阶段的第一个 3×3 卷积取出输入和输出，训练小网络去模仿它。四个阶段平均的测试损失，depthwise 卷积加 1×1 是 0.0148，分组卷积加 1×1 是 0.0133，PConv 加 1×1 是 0.0123，最接近原本的一般卷积。\n\n### FasterNet\n\n[![上半部分是 FasterNet 的整体架构，输入经过 Embedding 层，接着是四个阶段，每个阶段由若干个 FasterNet Block 组成，阶段之间以 Merging 层连接，特征图从 h/4×w/4 缩到 h/32×w/32，最后接全局池化、1×1 卷积和全连接层输出。下半部分左边放大说明部分卷积，输入只有前 c_p 个通道与 c_p 个 k×k 卷积核相乘，其余通道以 Identity 箭头直接接到输出；右边是 FasterNet Block，依次是 PConv 3×3、1×1 卷积、BN 与 ReLU、另一个 1×1 卷积，最后和输入相加](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)\n\n*原论文 Figure 4，取自 [**v3 PDF 第 5 页**](https://arxiv.org/pdf/2303.03667v3#page=5)，CC BY 4.0。*\n\nFasterNet 的骨架很朴素。stem 是步长 4 的 4×4 卷积，阶段之间用步长 2 的 2×2 卷积降采样。每个块是 PConv，接两个 1×1 卷积，中间把通道放大 2 倍，最后加上残差。\n\n有两个细节值得一提。一是归一化和激活函数只放在中间那一层之后，整个块只有一组；作者认为放太多会降低特征的多样性，也会拖慢速度。二是归一化用 BatchNorm，推理时可以合并进相邻的卷积。\n\n后两个阶段的内存访问比较少、FLOPS 比较高，所以作者把较多的块和运算放在后面，例如 T0 四个阶段的块数是 1、2、8、2。整个家族从小到大是 T0、T1、T2、S、M、L，小模型用 GELU，大模型用 ReLU。\n\n## 讨论\n\n### ImageNet\n\n作者在三种设备上测速度：GPU 是 2080Ti，批量 32 测吞吐量；CPU 是 i9-9900X，ARM 是 Cortex-A72，都只用单线程、批量 1 测延迟。以下摘几组准确率相近的模型：\n\n| 模型 | FLOPs | GPU（张/秒） | CPU（ms） | ARM（ms） | Top-1 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| MobileViT-XXS | 0.42G | 2,393 | 30.8 | 348 | 69.0% |\n| MobileNetV2 | 0.31G | 4,198 | 12.2 | 442 | 72.0% |\n| ShuffleNetV2 ×1.5 | 0.30G | 4,878 | 12.1 | 266 | **72.6%** |\n| **FasterNet-T0** | 0.34G | **6,807** | **9.2** | **143** | 71.9% |\n| ResNet-50 | 4.11G | 959 | 73.0 | 1,131 | 78.8% |\n| **FasterNet-T2** | 1.91G | **1,991** | **33.5** | **497** | **78.9%** |\n| Swin-T | 4.51G | 609 | 122.2 | 1,424 | 81.3% |\n| ConvNeXt-T | 4.47G | 657 | 86.3 | 1,889 | **82.1%** |\n| **FasterNet-S** | 4.56G | **1,029** | **71.2** | **1,103** | 81.3% |\n| Swin-B | 15.47G | 237 | 349.2 | OOM | 83.5% |\n| ConvNeXt-B | 15.38G | 253 | 257.1 | OOM | **83.8%** |\n| **FasterNet-L** | 15.52G | **323** | **219.5** | OOM | 83.5% |\n\n*表：摘自原论文 Table 3。MobileViT 的输入是 256×256，其余是 224×224；OOM 是内存不足。*\n\n每一组里，FasterNet 在三种设备上都最快。最好看的是最后一组，FasterNet-L 和 ConvNeXt-B、Swin-B 的 FLOPs 几乎一样，都在 15.4G 到 15.5G 左右，GPU 吞吐量却多了 28% 到 36%，CPU 时间也少了 15% 到 37%。FLOPs 一样、速度不一样，这就是整篇论文想说的事。\n\n准确率的部分就要仔细看了。摘要说 T0 比 MobileViT-XXS 准 2.9 个百分点，这是真的，可是同一组的 MobileNetV2 和 ShuffleNetV2 都比 T0 准。FasterNet-S 和 Swin-T 一样是 81.3%，比 ConvNeXt-T 低 0.8；FasterNet-L 论文说和 ConvNeXt-B「相当」，其实低了 0.3。FasterNet 换到的是速度，准确率大致持平，有时候还输一点。\n\n### 检测与分割\n\nCOCO 用 Mask R-CNN 训练 12 个 epoch。FasterNet-S 的 box AP 是 39.9，ResNet-50 是 38.0，GPU 延迟几乎一样（54.3 ms 对 54.9 ms）。FasterNet-L 是 44.0，PVT-Large 是 42.9，延迟则从 152.2 ms 降到 93.8 ms。\n\n### 消融\n\n| 设置 | GPU（张/秒） | CPU（ms） | ARM（ms） | Top-1 |\n| --- | ---: | ---: | ---: | ---: |\n| T0，$c_p = c/4$（默认） | 6,807 | 9.2 | 143 | 71.9% |\n| T0\\*，$c_p = c/2$ | 6,626 | 9.6 | 145 | 71.7% |\n| T0\\*，$c_p = c/8$ | 6,204 | 8.9 | 140 | 71.3% |\n| T0，LayerNorm 取代 BatchNorm | 5,515 | 10.7 | 159 | 71.9% |\n| T0，ReLU 取代 GELU | 6,929 | 8.2 | 114 | 71.3% |\n| T2，GELU 取代 ReLU | 1,985 | 35.4 | 557 | 78.7% |\n\n*表：摘自原论文 Table 5。T0\\* 是调整过宽度和深度、让复杂度相近的版本；T2 默认用 ReLU，所以这里比的是换成 GELU。*\n\n$c_p$ 取 1/4 最平衡；太大就退化成一般卷积，太小又抓不到空间特征。LayerNorm 和 BatchNorm 准确率一样，可是 LayerNorm 不能合并进卷积，GPU 吞吐量少了两成。激活函数则是小模型用 GELU 比较准，大模型用 ReLU 比较好，作者猜测是小模型更需要 GELU 的非线性。\n\n作者在附录里也写了两个局限。PConv 的步长只能是 1，否则卷积过的通道和没动的通道对不齐，降采样得交给另外的层。FasterNet 全部由小卷积组成，感受野有限，作者把扩大感受野留给以后的工作。\n\n## 结论\n\n延迟是 FLOPs 除以 FLOPS，这篇最值得记住的就是这个式子。只顾着压低分子，分母掉得更快，网络反而更慢；depthwise 卷积就是这样，运算省下来了，时间都花在搬内存上。\n\nPConv 简单到有点不好意思，切四分之一的通道做卷积，再接回去而已，几行就能搬进自己的网络。\n\n最后，速度一定要在自己的设备上测。论文用的是 2080Ti、i9-9900X 和 Cortex-A72，我们在 i9-13900K 上测到的趋势一样，数字却差不少。换成手机的 NPU 或别的推理框架，谁快谁慢都可能翻盘，别拿论文的表格直接下结论。\n","imageSizes":{"/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png":{"width":2083,"height":650},"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png":{"width":1012,"height":500},"/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png":{"width":1767,"height":742}}}