{"notes":[{"id":"lightweight/fasternet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FasterNet","subtitle":"FLOPs 會騙人","author":"Z. Yuan","paperDate":"2023-03-07","publishedOn":"2025-10-23","world":"external/fasternet","paperTitle":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","paperUrl":"https://arxiv.org/abs/2303.03667","bibliography":{"title":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","authors":["Jierun Chen","Shiu-hong Kao","Hao He","Weipeng Zhuo","Song Wen","Chul-Ho Lee","S.-H. Gary Chan"],"url":"https://arxiv.org/abs/2303.03667","pdfUrl":"https://arxiv.org/pdf/2303.03667v3","codeUrl":null,"preprint":{"id":"2303.03667","submittedOn":"2023-03-07","url":"https://arxiv.org/abs/2303.03667"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Chen_Run_Dont_Walk_Chasing_Higher_FLOPS_for_Faster_Neural_Networks_CVPR_2023_paper.html"},"citation":"@misc{lightweightfasternet2023,\n  title = {{Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks}},\n  author = {Jierun Chen and Shiu-hong Kao and Hao He and Weipeng Zhuo and Song Wen and Chul-Ho Lee and S.-H. Gary Chan},\n  year = {2023},\n  eprint = {2303.03667},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2303.03667}\n}\n","citationPath":"/content/papers/citations/lightweight/fasternet.7a313d25fdbe.bib"},"thumbnail":{"src":"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png","width":1012,"height":500},"locale":"zh-hant","path":"/content/papers/zh-hant/lightweight/fasternet.c8ec7f1bd55f.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"延遲等於-flops-除以-flops","text":"延遲等於 FLOPs 除以 FLOPS","depth":3},{"id":"depthwise-卷積慢在哪裡","text":"depthwise 卷積慢在哪裡","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"部分卷積-pconv","text":"部分卷積 PConv","depth":3},{"id":"再接一個-11-卷積","text":"再接一個 1×1 卷積","depth":3},{"id":"fasternet","text":"FasterNet","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"偵測與分割","text":"偵測與分割","depth":3},{"id":"消融","text":"消融","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/shufflenet","relation":"referenced","topic":""},{"id":"lightweight/ghostnet","relation":"referenced","topic":""}]},{"id":"lightweight/mobilenet-v2","category":"lightweight","legacyPath":"lightweight/1801-mobilenet-v2","legacySlug":"1801-mobilenet-v2","legacyPaths":[],"title":"MobileNet-V2","subtitle":"瓶頸內的淬煉","author":"Z. Yuan","paperDate":"2018-01-13","publishedOn":"2024-06-27","paperTitle":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","paperUrl":"https://arxiv.org/abs/1801.04381","bibliography":{"title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","authors":["Mark Sandler","Andrew Howard","Menglong Zhu","Andrey Zhmoginov","Liang-Chieh Chen"],"url":"https://arxiv.org/abs/1801.04381","pdfUrl":"https://arxiv.org/pdf/1801.04381","codeUrl":null,"preprint":{"id":"1801.04381","submittedOn":"2018-01-13","url":"https://arxiv.org/abs/1801.04381"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://arxiv.org/abs/1801.04381"},"citation":"@misc{lightweightmobilenetv22018,\n  title = {{MobileNetV2: Inverted Residuals and Linear Bottlenecks}},\n  author = {Mark Sandler and Andrew Howard and Menglong Zhu and Andrey Zhmoginov and Liang-Chieh Chen},\n  year = {2018},\n  eprint = {1801.04381},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.04381}\n}\n","citationPath":"/content/papers/citations/lightweight/mobilenet-v2.a882707cbaa2.bib"},"thumbnail":{"src":"/content/papers/assets/e6bbfb346bb449f83897d76b69c3711d2e9937d40320a7aab0f3843653a688c0.jpg","width":2332,"height":1324},"locale":"zh-hant","path":"/content/papers/zh-hant/lightweight/mobilenet-v2.a7a89cbedac8.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"線性瓶頸","text":"線性瓶頸","depth":3},{"id":"殘差網路","text":"殘差網路","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"反轉殘差","text":"反轉殘差","depth":3},{"id":"擴張瓶頸論文附錄-a","text":"擴張瓶頸（論文附錄 A）","depth":3},{"id":"模型架構","text":"模型架構","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"圖像分類效能","text":"圖像分類效能","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v1","relation":"background","topic":"行動裝置上的高效率卷積"},{"id":"lightweight/mobilenet-v3","relation":"next","topic":"行動裝置上的高效率卷積"}]},{"id":"lightweight/shufflenet","category":"lightweight","legacyPath":"lightweight/1707-shufflenet","legacySlug":"1707-shufflenet","legacyPaths":[],"title":"ShuffleNet","subtitle":"通道洗牌網路","author":"Z. Yuan","paperDate":"2017-07-04","publishedOn":"2024-06-26","paperTitle":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","paperUrl":"https://arxiv.org/abs/1707.01083","bibliography":{"title":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","authors":["Xiangyu Zhang","Xinyu Zhou","Mengxiao Lin","Jian Sun"],"url":"https://arxiv.org/abs/1707.01083","pdfUrl":"https://arxiv.org/pdf/1707.01083","codeUrl":null,"preprint":{"id":"1707.01083","submittedOn":"2017-07-04","url":"https://arxiv.org/abs/1707.01083"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://doi.org/10.1109/cvpr.2018.00716"},"citation":"@misc{lightweightshufflenet2017,\n  title = {{ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices}},\n  author = {Xiangyu Zhang and Xinyu Zhou and Mengxiao Lin and Jian Sun},\n  year = {2017},\n  eprint = {1707.01083},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1707.01083}\n}\n","citationPath":"/content/papers/citations/lightweight/shufflenet.1e5a6869f5a9.bib"},"thumbnail":{"src":"/content/papers/assets/6d0c746d403d2823809ffbd52c9b3614cc2063afee07608d6d0cb26b11fc3ac8.jpg","width":1746,"height":678},"locale":"zh-hant","path":"/content/papers/zh-hant/lightweight/shufflenet.c6ae9d7934e2.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"計算量與速度不一致","text":"計算量與速度不一致","depth":3},{"id":"大量-1x1-卷積","text":"大量 1x1 卷積","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"組卷積","text":"組卷積","depth":3},{"id":"通道洗牌","text":"通道洗牌","depth":3},{"id":"洗牌模組放置位置","text":"洗牌模組放置位置","depth":3},{"id":"模型架構","text":"模型架構","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"組卷積的效果","text":"組卷積的效果","depth":3},{"id":"通道洗牌的效果","text":"通道洗牌的效果","depth":3},{"id":"與其他模型的比較","text":"與其他模型的比較","depth":3},{"id":"真實推論速度評估","text":"真實推論速度評估","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/ghostnet","relation":"comparison","topic":"降低卷積與通道的運算成本"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"降低卷積與通道的運算成本"}]},{"id":"lightweight/ghostnet","category":"lightweight","legacyPath":"lightweight/1911-ghostnet","legacySlug":"1911-ghostnet","legacyPaths":[],"title":"GhostNet","subtitle":"特徵圖中的幽靈","author":"Z. Yuan","paperDate":"2019-11-27","publishedOn":"2024-07-23","paperTitle":"GhostNet: More Features from Cheap Operations","paperUrl":"https://arxiv.org/abs/1911.11907","bibliography":{"title":"GhostNet: More Features from Cheap Operations","authors":["Kai Han","Yunhe Wang","Qi Tian","Jianyuan Guo","Chunjing Xu","Chang Xu"],"url":"https://arxiv.org/abs/1911.11907","pdfUrl":"https://arxiv.org/pdf/1911.11907","codeUrl":"https://github.com/huawei-noah/ghostnet","preprint":{"id":"1911.11907","submittedOn":"2019-11-27","url":"https://arxiv.org/abs/1911.11907"},"publication":{"venue":"CVPR","year":2020,"kind":"conference","url":"https://arxiv.org/abs/1911.11907"},"citation":"@misc{lightweightghostnet2019,\n  title = {{GhostNet: More Features from Cheap Operations}},\n  author = {Kai Han and Yunhe Wang and Qi Tian and Jianyuan Guo and Chunjing Xu and Chang Xu},\n  year = {2019},\n  eprint = {1911.11907},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1911.11907}\n}\n","citationPath":"/content/papers/citations/lightweight/ghostnet.4dd65d9540bb.bib"},"thumbnail":{"src":"/content/papers/assets/22fa9d36b042deb7995e9695a803de3a973542f58cd9cd48e37ef13167df2222.jpg","width":972,"height":904},"locale":"zh-hant","path":"/content/papers/zh-hant/lightweight/ghostnet.39b24623ded8.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"模組配置","text":"模組配置","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet-上的性能","text":"ImageNet 上的性能","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/shufflenet","relation":"comparison","topic":"降低卷積與通道的運算成本"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"降低卷積與通道的運算成本"}]}],"text":"設計輕量網路的人，最常看的數字是 FLOPs。MobileNet、ShuffleNet、GhostNet 一路下來，大家想盡辦法把浮點運算量壓低，最常用的工具是 depthwise 卷積和分組卷積。\n\n\n> [!TIP]\n> \n> 這幾篇可以先看我們的筆記：\n> \n> - **[[18.01] MobileNet-V2：瓶頸內的淬煉](/blog/papers/lightweight/mobilenet-v2)**\n> - **[[17.07] ShuffleNet：通道洗牌網路](/blog/papers/lightweight/shufflenet)**\n> - **[[19.11] GhostNet：特徵圖中的幽靈](/blog/papers/lightweight/ghostnet)**\n\n\n可是 FLOPs 少了，網路跑得就一定快嗎？香港科技大學的 Jierun Chen 等人量了一圈，發現答案常常是否定的。例如 CycleMLP-B1 的 FLOPs 只有 ResNet-50 的一半，在 CPU 上卻比較慢，116.1 ms 對 73.0 ms。這篇的標題叫「Run, Don't Walk」，意思是與其省步數，不如跑快一點。\n\n## 定義問題\n\n### 延遲等於 FLOPs 除以 FLOPS\n\n這兩個字只差一個大小寫。FLOPs 是浮點運算的次數，FLOPS 是每秒能做多少次浮點運算，也就是硬體實際跑出來的速度。延遲是兩者相除：\n\n$$\n\\text{Latency}=\\frac{\\text{FLOPs}}{\\text{FLOPS}} .\n$$\n\n只壓低 FLOPs，如果 FLOPS 掉得更多，延遲反而會變長。作者在 Intel CPU 上量了一批常見的網路：\n\n[![兩張圖。(a) 橫軸是 FLOPs（G），縱軸是 CPU 上實際的 FLOPS（G/s）。橘色星號的 FasterNet 在最上方，從約 37 一路升到 64；ResNet50 是一個約 56 的單點；PVT、PoolFormer、MobileViT、EdgeNeXt 大約落在 13 到 43 之間；MobileNetV2、GhostNet、ShuffleNetV2、MobileFormer、CycleMLP 多數在 30 以下，MobileFormer 和 GhostNet 最低，只有個位數到十幾。(b) 橫軸是 FLOPs，縱軸是 CPU 延遲（ms），右邊是完整範圍，左邊放大 1.8 GFLOPs 以下的部分；同樣的 FLOPs 下，FasterNet 的橘線都在最下方，CycleMLP 的兩個點在最上方，約 116 和 186 ms](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)\n\n*原論文 Figure 2，取自 [**v3 PDF 第 2 頁**](https://arxiv.org/pdf/2303.03667v3#page=2)，CC BY 4.0。(a) CPU 上的 FLOPS 對 FLOPs；(b) CPU 延遲對 FLOPs。*\n\n左圖的縱軸是每秒實際做了多少運算。許多號稱快速的網路，FLOPS 都比 ResNet-50 低，硬體的運算能力沒有用上。\n\n這個現象 ShuffleNet V2 在 2018 年就提過，它也指出 FLOPs 和速度對不上。可是之後的輕量網路還是繼續用 depthwise 卷積，因為一直沒有更好的替代品。\n\n### depthwise 卷積慢在哪裡\n\n輸入是 $c\\times h\\times w$ 的特徵圖，3×3 卷積核的邊長 $k=3$。一般卷積的 FLOPs 是 $h\\times w\\times k^2\\times c^2$，depthwise 卷積每個通道只用自己的卷積核，FLOPs 降到 $h\\times w\\times k^2\\times c$，少了 $c$ 倍。\n\n問題在於 depthwise 卷積只做空間方向的運算，單獨用的話準確率掉很多，所以實際上都會把通道數放大，MobileNet-V2 的倒殘差區塊就放大了 6 倍。通道從 $c$ 變成 $c'$ 之後，記憶體存取量是\n\n$$\nh\\times w\\times 2c' + k^2\\times c' \\approx h\\times w\\times 2c' ,\n$$\n\n讀一次輸入、寫一次輸出，幾乎全是特徵圖的搬運。一般卷積的存取量約是 $h\\times w\\times 2c$，反而比較少。\n\n運算少、搬運多，每搬一個數只做幾次運算，硬體就會一直在等記憶體。\n\n## 解決問題\n\n### 部分卷積 PConv\n\n作者觀察到，特徵圖的不同通道之間常常很像。他們把預訓練 ResNet-50 中間層的特徵圖畫出來，很多通道看起來非常相似。既然這麼重複，空間方向的卷積就不必每個通道都做。\n\n[![三種卷積的示意圖。(a) 一般卷積：一個長方體輸入和一整排卷積核做卷積，得到同樣大小的輸出。(b) depthwise 或分組卷積：輸入沿通道方向被虛線切成很多片，每片各自配一個小卷積核。(c) 部分卷積：輸入只有前面一小段通道用虛線框出來，配上少數幾個卷積核做卷積，剩下的通道沿著一條箭頭原封不動地接到輸出](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)\n\n*原論文 Figure 1，取自 [**v3 PDF 第 1 頁**](https://arxiv.org/pdf/2303.03667v3#page=1)，CC BY 4.0。*\n\n部分卷積（Partial Convolution，PConv）只對前 $c_p$ 個通道做一般的 3×3 卷積，剩下的通道直接複製到輸出。取前面連續的一段，是為了讓記憶體存取保持連續。預設 $c_p = c/4$，FLOPs 是\n\n$$\nh\\times w\\times k^2\\times c_p^2 ,\n$$\n\n只有一般卷積的 1/16；記憶體存取量約 $h\\times w\\times 2c_p$，是一般卷積的 1/4。\n\n我們在自己的 CPU 上量量看。三種運算子各疊 10 層，輸入是 192×28×28，另外補一組把通道放大 6 倍的 depthwise 卷積，模擬 MobileNet-V2 的用法：\n\n```python\nimport time\nimport torch\nimport torch.nn as nn\n\ntorch.set_num_threads(1)                                   # 和論文一樣，CPU 只用單執行緒\nc, h, w = 192, 28, 28                                      # 第二階段常見的特徵圖大小\ncp = c // 4                                                # PConv 只對 1/4 的通道做卷積\n\nclass PConv(nn.Module):\n    def __init__(self, c, cp):\n        super().__init__()\n        self.cp = cp\n        self.conv = nn.Conv2d(cp, cp, 3, padding=1, bias=False)\n\n    def forward(self, x):\n        x1, x2 = torch.split(x, [self.cp, x.shape[1] - self.cp], dim=1)\n        return torch.cat((self.conv(x1), x2), 1)           # 其餘 3/4 的通道原封不動\n\n# (運算子, 輸入通道數, FLOPs, 記憶體存取量)；FLOPs 以乘加次數計，存取量是讀寫特徵圖加上讀權重的元素數\nops = {\n    \"Conv 3×3\": (nn.Conv2d(c, c, 3, padding=1, bias=False), c, h * w * 9 * c * c, h * w * 2 * c + 9 * c * c),\n    \"DWConv 3×3\": (nn.Conv2d(c, c, 3, padding=1, groups=c, bias=False), c, h * w * 9 * c, h * w * 2 * c + 9 * c),\n    \"PConv 3×3\": (PConv(c, cp), c, h * w * 9 * cp * cp, h * w * 2 * cp + 9 * cp * cp),\n    \"DWConv ×6\": (nn.Conv2d(6 * c, 6 * c, 3, padding=1, groups=6 * c, bias=False), 6 * c,\n                  h * w * 9 * 6 * c, h * w * 2 * 6 * c + 9 * 6 * c),\n}\nwith torch.no_grad():\n    for name, (op, cin, flops, mem) in ops.items():\n        x = torch.randn(1, cin, h, w)\n        net = nn.Sequential(*[op] * 10)                    # 同一層疊 10 次\n        for _ in range(20):\n            net(x)                                         # 暖身\n        t = time.perf_counter()\n        for _ in range(200):\n            net(x)\n        ms = (time.perf_counter() - t) / 200 * 1000\n        total = flops * 10\n        print(f\"{name:<10}  FLOPs {total / 1e6:6.1f}M  每存取一個數做 {flops / mem:5.1f} 次乘加  \"\n              f\"延遲 {ms:5.2f} ms  FLOPS {total / ms / 1e6:4.1f} G/s\")\n```\n\n```\nConv 3×3    FLOPs 2601.1M  每存取一個數做 411.0 次乘加  延遲 32.83 ms  FLOPS 79.2 G/s\nDWConv 3×3  FLOPs   13.5M  每存取一個數做   4.5 次乘加  延遲  0.83 ms  FLOPS 16.4 G/s\nPConv 3×3   FLOPs  162.6M  每存取一個數做 169.3 次乘加  延遲  2.72 ms  FLOPS 59.8 G/s\nDWConv ×6   FLOPs   81.3M  每存取一個數做   4.5 次乘加  延遲 10.29 ms  FLOPS  7.9 G/s\n```\n\n這是在 Intel i9-13900K 上跑的結果，換一台機器數字會不同，趨勢應該差不多。\n\n先看 FLOPS 那一欄。一般卷積每存取一個數要做 411 次乘加，硬體用得最滿；depthwise 卷積只做 4.5 次，FLOPS 掉到五分之一左右。PConv 每個數做 169 次，FLOPS 大約是一般卷積的四分之三。\n\n公平地說，同樣 192 個通道、同樣疊 10 層，不加寬的 depthwise 卷積其實最快，只要 0.83 ms，因為它的 FLOPs 實在太少了。可是它不會單獨這樣用。通道放大 6 倍之後，FLOPs 只有 PConv 的一半，延遲卻是 PConv 的將近 4 倍。\n\n論文用 2080Ti、i9-9900X 和 Cortex-A72 量了同樣的實驗，四種特徵圖大小平均起來，PConv 的 FLOPs 是一般卷積的 1/16，FLOPS 則是 depthwise 卷積的 10.5 倍（GPU）、6.2 倍（CPU）和 22.8 倍（ARM）。\n\n### 再接一個 1×1 卷積\n\nPConv 只看了四分之一的通道，剩下的資訊要靠後面的 1×1 卷積補上。兩者合起來的感受野像一個 T 字：中心那個位置看了所有通道，周圍的八個位置只看了 $c_p$ 個通道。\n\n作者用預訓練的 ResNet-18 檢查這個形狀合不合理。對每個 3×3 卷積核，算九個位置各自的權重大小，看哪個位置最大；四個階段裡，最常是最大的都是正中間那個位置。中心本來就比周圍重要，多花一點運算在中心上是合理的。\n\n那為什麼不直接用一個 T 字形的卷積？因為拆開比較省。T 字形卷積的 FLOPs 是 $h\\times w\\times(k^2 c_p c + c(c-c_p))$，拆成 PConv 加 1×1 卷積是 $h\\times w\\times(k^2 c_p^2 + c^2)$。$c_p = c/4$ 時，前者是 $3c^2$，後者只有 $1.5625c^2$。\n\n作者也驗證了這個組合能不能取代一般卷積。從預訓練 ResNet-50 每個階段的第一個 3×3 卷積取出輸入和輸出，訓練小網路去模仿它。四個階段平均的測試損失，depthwise 卷積加 1×1 是 0.0148，分組卷積加 1×1 是 0.0133，PConv 加 1×1 是 0.0123，最接近原本的一般卷積。\n\n### FasterNet\n\n[![上半部是 FasterNet 的整體架構，輸入經過 Embedding 層，接著是四個階段，每個階段由若干個 FasterNet Block 組成，階段之間以 Merging 層連接，特徵圖從 h/4×w/4 縮到 h/32×w/32，最後接全域池化、1×1 卷積和全連接層輸出。下半部左邊放大說明部分卷積，輸入只有前 c_p 個通道與 c_p 個 k×k 卷積核相乘，其餘通道以 Identity 箭頭直接接到輸出；右邊是 FasterNet Block，依序是 PConv 3×3、1×1 卷積、BN 與 ReLU、另一個 1×1 卷積，最後和輸入相加](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)\n\n*原論文 Figure 4，取自 [**v3 PDF 第 5 頁**](https://arxiv.org/pdf/2303.03667v3#page=5)，CC BY 4.0。*\n\nFasterNet 的骨架很樸素。stem 是步長 4 的 4×4 卷積，階段之間用步長 2 的 2×2 卷積降採樣。每個區塊是 PConv，接兩個 1×1 卷積，中間把通道放大 2 倍，最後加上殘差。\n\n有兩個細節值得一提。一是正規化和激活函數只放在中間那一層之後，整個區塊只有一組；作者認為放太多會降低特徵的多樣性，也會拖慢速度。二是正規化用 BatchNorm，推論時可以併進相鄰的卷積。\n\n後兩個階段的記憶體存取比較少、FLOPS 比較高，所以作者把較多的區塊和運算放在後面，例如 T0 四個階段的區塊數是 1、2、8、2。整個家族從小到大是 T0、T1、T2、S、M、L，小模型用 GELU，大模型用 ReLU。\n\n## 討論\n\n### ImageNet\n\n作者在三種裝置上量速度：GPU 是 2080Ti，批次 32 量吞吐量；CPU 是 i9-9900X，ARM 是 Cortex-A72，都只用單執行緒、批次 1 量延遲。以下摘幾組準確率相近的模型：\n\n| 模型 | FLOPs | GPU（張／秒） | CPU（ms） | ARM（ms） | Top-1 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| MobileViT-XXS | 0.42G | 2,393 | 30.8 | 348 | 69.0% |\n| MobileNetV2 | 0.31G | 4,198 | 12.2 | 442 | 72.0% |\n| ShuffleNetV2 ×1.5 | 0.30G | 4,878 | 12.1 | 266 | **72.6%** |\n| **FasterNet-T0** | 0.34G | **6,807** | **9.2** | **143** | 71.9% |\n| ResNet-50 | 4.11G | 959 | 73.0 | 1,131 | 78.8% |\n| **FasterNet-T2** | 1.91G | **1,991** | **33.5** | **497** | **78.9%** |\n| Swin-T | 4.51G | 609 | 122.2 | 1,424 | 81.3% |\n| ConvNeXt-T | 4.47G | 657 | 86.3 | 1,889 | **82.1%** |\n| **FasterNet-S** | 4.56G | **1,029** | **71.2** | **1,103** | 81.3% |\n| Swin-B | 15.47G | 237 | 349.2 | OOM | 83.5% |\n| ConvNeXt-B | 15.38G | 253 | 257.1 | OOM | **83.8%** |\n| **FasterNet-L** | 15.52G | **323** | **219.5** | OOM | 83.5% |\n\n*表：摘自原論文 Table 3。MobileViT 的輸入是 256×256，其餘是 224×224；OOM 是記憶體不足。*\n\n每一組裡，FasterNet 在三種裝置上都最快。最好看的是最後一組，FasterNet-L 和 ConvNeXt-B、Swin-B 的 FLOPs 幾乎一樣，都是 15.4G 到 15.5G 左右，GPU 吞吐量卻多了 28% 到 36%，CPU 時間也少了 15% 到 37%。FLOPs 一樣、速度不一樣，這就是整篇論文想說的事。\n\n準確率的部分就要仔細看了。摘要說 T0 比 MobileViT-XXS 準 2.9 個百分點，這是真的，可是同一組的 MobileNetV2 和 ShuffleNetV2 都比 T0 準。FasterNet-S 和 Swin-T 一樣是 81.3%，比 ConvNeXt-T 低 0.8；FasterNet-L 論文說和 ConvNeXt-B「相當」，其實低了 0.3。FasterNet 換到的是速度，準確率大致持平，有時候還輸一點。\n\n### 偵測與分割\n\nCOCO 用 Mask R-CNN 訓練 12 個 epoch。FasterNet-S 的 box AP 是 39.9，ResNet-50 是 38.0，GPU 延遲幾乎一樣（54.3 ms 對 54.9 ms）。FasterNet-L 是 44.0，PVT-Large 是 42.9，延遲則從 152.2 ms 降到 93.8 ms。\n\n### 消融\n\n| 設定 | GPU（張／秒） | CPU（ms） | ARM（ms） | Top-1 |\n| --- | ---: | ---: | ---: | ---: |\n| T0，$c_p = c/4$（預設） | 6,807 | 9.2 | 143 | 71.9% |\n| T0\\*，$c_p = c/2$ | 6,626 | 9.6 | 145 | 71.7% |\n| T0\\*，$c_p = c/8$ | 6,204 | 8.9 | 140 | 71.3% |\n| T0，LayerNorm 取代 BatchNorm | 5,515 | 10.7 | 159 | 71.9% |\n| T0，ReLU 取代 GELU | 6,929 | 8.2 | 114 | 71.3% |\n| T2，GELU 取代 ReLU | 1,985 | 35.4 | 557 | 78.7% |\n\n*表：摘自原論文 Table 5。T0\\* 是調整過寬度和深度、讓複雜度相近的版本；T2 預設用 ReLU，所以這裡比的是換成 GELU。*\n\n$c_p$ 取 1/4 最平衡；太大就退化成一般卷積，太小又抓不到空間特徵。LayerNorm 和 BatchNorm 準確率一樣，可是 LayerNorm 不能併進卷積，GPU 吞吐量少了兩成。激活函數則是小模型用 GELU 比較準，大模型用 ReLU 比較好，作者猜是小模型更需要 GELU 的非線性。\n\n作者在附錄也寫了兩個限制。PConv 的步長只能是 1，否則卷積過的通道和沒動的通道對不齊，降採樣得交給另外的層。FasterNet 全部由小卷積組成，感受野有限，作者把擴大感受野留給以後的工作。\n\n## 結論\n\n延遲是 FLOPs 除以 FLOPS，這篇最值得記住的就是這個式子。只顧著壓低分子，分母掉得更快，網路反而更慢；depthwise 卷積就是這樣，運算省下來了，時間都花在搬記憶體上。\n\nPConv 簡單到有點不好意思，切四分之一的通道做卷積，再接回去而已，幾行就能搬進自己的網路。\n\n最後，速度一定要在自己的裝置上量。論文用的是 2080Ti、i9-9900X 和 Cortex-A72，我們在 i9-13900K 上量到的趨勢一樣，數字卻差不少。換成手機的 NPU 或別的推論框架，誰快誰慢都可能翻盤，別拿論文的表格直接下結論。\n","imageSizes":{"/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png":{"width":2083,"height":650},"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png":{"width":1012,"height":500},"/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png":{"width":1767,"height":742}}}