{"notes":[{"id":"lightweight/fasternet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FasterNet","subtitle":"FLOPs Lie","author":"Z. Yuan","paperDate":"2023-03-07","publishedOn":"2025-10-23","world":"external/fasternet","paperTitle":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","paperUrl":"https://arxiv.org/abs/2303.03667","bibliography":{"title":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","authors":["Jierun Chen","Shiu-hong Kao","Hao He","Weipeng Zhuo","Song Wen","Chul-Ho Lee","S.-H. Gary Chan"],"url":"https://arxiv.org/abs/2303.03667","pdfUrl":"https://arxiv.org/pdf/2303.03667v3","codeUrl":null,"preprint":{"id":"2303.03667","submittedOn":"2023-03-07","url":"https://arxiv.org/abs/2303.03667"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Chen_Run_Dont_Walk_Chasing_Higher_FLOPS_for_Faster_Neural_Networks_CVPR_2023_paper.html"},"citation":"@misc{lightweightfasternet2023,\n  title = {{Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks}},\n  author = {Jierun Chen and Shiu-hong Kao and Hao He and Weipeng Zhuo and Song Wen and Chul-Ho Lee and S.-H. Gary Chan},\n  year = {2023},\n  eprint = {2303.03667},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2303.03667}\n}\n","citationPath":"/content/papers/citations/lightweight/fasternet.7a313d25fdbe.bib"},"thumbnail":{"src":"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png","width":1012,"height":500},"locale":"en","path":"/content/papers/en/lightweight/fasternet.c960f39f4c31.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"latency-is-flops-divided-by-flops","text":"Latency Is FLOPs Divided by FLOPS","depth":3},{"id":"why-depthwise-convolution-is-slow","text":"Why Depthwise Convolution Is Slow","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"partial-convolution-pconv","text":"Partial Convolution (PConv)","depth":3},{"id":"followed-by-a-11-convolution","text":"Followed by a 1×1 Convolution","depth":3},{"id":"fasternet","text":"FasterNet","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"ablation","text":"Ablation","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/shufflenet","relation":"referenced","topic":""},{"id":"lightweight/ghostnet","relation":"referenced","topic":""}]},{"id":"lightweight/mobilenet-v2","category":"lightweight","legacyPath":"lightweight/1801-mobilenet-v2","legacySlug":"1801-mobilenet-v2","legacyPaths":[],"title":"MobileNet-V2","subtitle":"Refining the Bottleneck","author":"Z. Yuan","paperDate":"2018-01-13","publishedOn":"2024-06-27","paperTitle":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","paperUrl":"https://arxiv.org/abs/1801.04381","bibliography":{"title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","authors":["Mark Sandler","Andrew Howard","Menglong Zhu","Andrey Zhmoginov","Liang-Chieh Chen"],"url":"https://arxiv.org/abs/1801.04381","pdfUrl":"https://arxiv.org/pdf/1801.04381","codeUrl":null,"preprint":{"id":"1801.04381","submittedOn":"2018-01-13","url":"https://arxiv.org/abs/1801.04381"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://arxiv.org/abs/1801.04381"},"citation":"@misc{lightweightmobilenetv22018,\n  title = {{MobileNetV2: Inverted Residuals and Linear Bottlenecks}},\n  author = {Mark Sandler and Andrew Howard and Menglong Zhu and Andrey Zhmoginov and Liang-Chieh Chen},\n  year = {2018},\n  eprint = {1801.04381},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.04381}\n}\n","citationPath":"/content/papers/citations/lightweight/mobilenet-v2.a882707cbaa2.bib"},"thumbnail":{"src":"/content/papers/assets/e6bbfb346bb449f83897d76b69c3711d2e9937d40320a7aab0f3843653a688c0.jpg","width":2332,"height":1324},"locale":"en","path":"/content/papers/en/lightweight/mobilenet-v2.db02333fcec2.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"nonlinearity-in-narrow-channels","text":"Nonlinearity in Narrow Channels","depth":3},{"id":"where-does-the-residual-shortcut-connect","text":"Where Does the Residual Shortcut Connect?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"inverted-residual-and-linear-bottleneck","text":"Inverted Residual and Linear Bottleneck","depth":3},{"id":"why-expand-the-bottleneck","text":"Why Expand the Bottleneck?","depth":3},{"id":"from-block-to-network","text":"From Block to Network","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet-classification-and-phone-latency","text":"ImageNet Classification and Phone Latency","depth":3},{"id":"other-tasks-and-limits","text":"Other Tasks and Limits","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v1","relation":"background","topic":"Efficient convolution on mobile devices"},{"id":"lightweight/mobilenet-v3","relation":"next","topic":"Efficient convolution on mobile devices"}]},{"id":"lightweight/shufflenet","category":"lightweight","legacyPath":"lightweight/1707-shufflenet","legacySlug":"1707-shufflenet","legacyPaths":[],"title":"ShuffleNet","subtitle":"Channel Shuffling Network","author":"Z. Yuan","paperDate":"2017-07-04","publishedOn":"2024-06-26","paperTitle":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","paperUrl":"https://arxiv.org/abs/1707.01083","bibliography":{"title":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","authors":["Xiangyu Zhang","Xinyu Zhou","Mengxiao Lin","Jian Sun"],"url":"https://arxiv.org/abs/1707.01083","pdfUrl":"https://arxiv.org/pdf/1707.01083","codeUrl":null,"preprint":{"id":"1707.01083","submittedOn":"2017-07-04","url":"https://arxiv.org/abs/1707.01083"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://doi.org/10.1109/cvpr.2018.00716"},"citation":"@misc{lightweightshufflenet2017,\n  title = {{ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices}},\n  author = {Xiangyu Zhang and Xinyu Zhou and Mengxiao Lin and Jian Sun},\n  year = {2017},\n  eprint = {1707.01083},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1707.01083}\n}\n","citationPath":"/content/papers/citations/lightweight/shufflenet.1e5a6869f5a9.bib"},"thumbnail":{"src":"/content/papers/assets/6d0c746d403d2823809ffbd52c9b3614cc2063afee07608d6d0cb26b11fc3ac8.jpg","width":1746,"height":678},"locale":"en","path":"/content/papers/en/lightweight/shufflenet.faaefe20498b.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"computation-and-speed-do-not-always-match","text":"Computation and Speed Do Not Always Match","depth":3},{"id":"the-cost-of-11-convolutions","text":"The Cost of 1×1 Convolutions","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"group-convolution-and-cross-group-flow","text":"Group Convolution and Cross-Group Flow","depth":3},{"id":"channel-shuffle","text":"Channel Shuffle","depth":3},{"id":"where-does-the-shuffle-go","text":"Where Does the Shuffle Go?","depth":3},{"id":"network-architecture","text":"Network Architecture","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"effect-of-group-convolution","text":"Effect of Group Convolution","depth":3},{"id":"effect-of-channel-shuffle","text":"Effect of Channel Shuffle","depth":3},{"id":"comparison-with-mobilenet","text":"Comparison with MobileNet","depth":3},{"id":"actual-inference-speed","text":"Actual Inference Speed","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/ghostnet","relation":"comparison","topic":"Reducing convolution and channel cost"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"Reducing convolution and channel cost"}]},{"id":"lightweight/ghostnet","category":"lightweight","legacyPath":"lightweight/1911-ghostnet","legacySlug":"1911-ghostnet","legacyPaths":[],"title":"GhostNet","subtitle":"Ghost in Feature Maps","author":"Z. Yuan","paperDate":"2019-11-27","publishedOn":"2024-07-23","paperTitle":"GhostNet: More Features from Cheap Operations","paperUrl":"https://arxiv.org/abs/1911.11907","bibliography":{"title":"GhostNet: More Features from Cheap Operations","authors":["Kai Han","Yunhe Wang","Qi Tian","Jianyuan Guo","Chunjing Xu","Chang Xu"],"url":"https://arxiv.org/abs/1911.11907","pdfUrl":"https://arxiv.org/pdf/1911.11907","codeUrl":"https://github.com/huawei-noah/ghostnet","preprint":{"id":"1911.11907","submittedOn":"2019-11-27","url":"https://arxiv.org/abs/1911.11907"},"publication":{"venue":"CVPR","year":2020,"kind":"conference","url":"https://arxiv.org/abs/1911.11907"},"citation":"@misc{lightweightghostnet2019,\n  title = {{GhostNet: More Features from Cheap Operations}},\n  author = {Kai Han and Yunhe Wang and Qi Tian and Jianyuan Guo and Chunjing Xu and Chang Xu},\n  year = {2019},\n  eprint = {1911.11907},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1911.11907}\n}\n","citationPath":"/content/papers/citations/lightweight/ghostnet.4dd65d9540bb.bib"},"thumbnail":{"src":"/content/papers/assets/22fa9d36b042deb7995e9695a803de3a973542f58cd9cd48e37ef13167df2222.jpg","width":972,"height":904},"locale":"en","path":"/content/papers/en/lightweight/ghostnet.fb6ae4b0873d.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"model-architecture","text":"Model Architecture","depth":3},{"id":"module-configuration","text":"Module Configuration","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"performance-on-imagenet","text":"Performance on ImageNet","depth":3},{"id":"actual-inference-speed","text":"Actual Inference Speed","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/shufflenet","relation":"comparison","topic":"Reducing convolution and channel cost"},{"id":"lightweight/pp-lcnet","relation":"comparison","topic":"Reducing convolution and channel cost"}]}],"text":"The number people designing lightweight networks look at most is FLOPs. From MobileNet to ShuffleNet to GhostNet, everyone did all they could to push the count of floating-point operations down, and the most common tools were depthwise convolution and grouped convolution.\n\n\n> [!TIP]\n> \n> For these, you can start with our notes:\n> \n> - **[[18.01] MobileNet-V2: Refining the Bottleneck](/blog/papers/lightweight/mobilenet-v2)**\n> - **[[17.07] ShuffleNet: Channel Shuffling Network](/blog/papers/lightweight/shufflenet)**\n> - **[[19.11] GhostNet: Ghost in Feature Maps](/blog/papers/lightweight/ghostnet)**\n\n\nBut with fewer FLOPs, does a network necessarily run faster? Jierun Chen and colleagues at the Hong Kong University of Science and Technology measured a whole range of networks and found the answer is often no. CycleMLP-B1, for example, has only half the FLOPs of ResNet-50, yet it's slower on a CPU, 116.1 ms versus 73.0 ms. The paper's title, “Run, Don't Walk,” means that rather than saving steps, you're better off running faster.\n\n## Defining the Problem\n\n### Latency Is FLOPs Divided by FLOPS\n\nThese two words differ only in the case of one letter. FLOPs is the number of floating-point operations, while FLOPS is how many floating-point operations can be done per second, that is, the speed the hardware actually delivers. Latency is one divided by the other:\n\n$$\n\\text{Latency}=\\frac{\\text{FLOPs}}{\\text{FLOPS}} .\n$$\n\nIf you only push FLOPs down and FLOPS falls even further, latency actually gets longer. The authors measured a batch of common networks on an Intel CPU:\n\n[![Two charts. (a) The horizontal axis is FLOPs (G) and the vertical axis is the actual FLOPS on CPU (G/s). FasterNet's orange stars are at the top, rising from about 37 all the way to 64; ResNet50 is a single point at about 56; PVT, PoolFormer, MobileViT, and EdgeNeXt fall roughly between 13 and 43; MobileNetV2, GhostNet, ShuffleNetV2, MobileFormer, and CycleMLP are mostly below 30, with MobileFormer and GhostNet the lowest, from single digits to the teens. (b) The horizontal axis is FLOPs and the vertical axis is CPU latency (ms), with the full range on the right and a zoom on the part below 1.8 GFLOPs on the left; at the same FLOPs, FasterNet's orange line is always at the bottom, and CycleMLP's two points are at the top, at about 116 and 186 ms](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)](/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png)\n\n*Figure 2 from the paper, excerpted from [**page 2 of the v3 PDF**](https://arxiv.org/pdf/2303.03667v3#page=2), CC BY 4.0. (a) FLOPS versus FLOPs on CPU; (b) CPU latency versus FLOPs.*\n\nThe vertical axis of the left chart is how many operations are actually done per second. Many networks touted as fast have lower FLOPS than ResNet-50, leaving the hardware's compute unused.\n\nShuffleNet V2 already raised this in 2018, likewise pointing out that FLOPs and speed don't line up. Later lightweight networks kept using depthwise convolution anyway, because no better replacement had come along.\n\n### Why Depthwise Convolution Is Slow\n\nTake an input feature map of $c\\times h\\times w$ and a 3×3 kernel with side length $k=3$. Ordinary convolution has $h\\times w\\times k^2\\times c^2$ FLOPs. Depthwise convolution uses only each channel's own kernel, bringing FLOPs down to $h\\times w\\times k^2\\times c$, $c$ times fewer.\n\nThe problem is that depthwise convolution only computes along the spatial dimensions, and used on its own it loses a lot of accuracy, so in practice the channel count always gets expanded; MobileNet-V2's inverted residual block expands it 6×. Once the channels go from $c$ to $c'$, the memory access is\n\n$$\nh\\times w\\times 2c' + k^2\\times c' \\approx h\\times w\\times 2c' ,\n$$\n\none read of the input and one write of the output, nearly all of it spent moving feature maps around. Ordinary convolution's memory access is about $h\\times w\\times 2c$, which is actually less.\n\nWith little computation, lots of data movement, and only a few operations for every number moved, the hardware spends its time waiting on memory.\n\n## Solving the Problem\n\n### Partial Convolution (PConv)\n\nThe authors observe that the different channels of a feature map are often very similar. They plotted the feature maps of an intermediate layer of a pretrained ResNet-50, and many channels looked very much alike. With that much repetition, the spatial convolution doesn't have to be done on every channel.\n\n[![Diagram of three kinds of convolution. (a) Ordinary convolution: a cuboid input is convolved with a full row of kernels, producing an output of the same size. (b) Depthwise or grouped convolution: the input is cut by dashed lines into many slices along the channel direction, each slice with its own small kernel. (c) Partial convolution: only a short stretch of channels at the front of the input is outlined with dashed lines and convolved with a few kernels, while the remaining channels are carried unchanged to the output along an arrow](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)](/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png)\n\n*Figure 1 from the paper, excerpted from [**page 1 of the v3 PDF**](https://arxiv.org/pdf/2303.03667v3#page=1), CC BY 4.0.*\n\nPartial Convolution (PConv) applies an ordinary 3×3 convolution only to the first $c_p$ channels and copies the remaining channels straight to the output. Taking a contiguous stretch at the front keeps memory access contiguous. By default $c_p = c/4$, and the FLOPs are\n\n$$\nh\\times w\\times k^2\\times c_p^2 ,\n$$\n\nonly 1/16 of ordinary convolution; the memory access is about $h\\times w\\times 2c_p$, 1/4 of ordinary convolution.\n\nLet's measure it on our own CPU. Each of three operators is stacked 10 layers deep on a 192×28×28 input, plus one extra group of depthwise convolution with the channels expanded 6× to mimic how MobileNet-V2 uses it:\n\n```python\nimport time\nimport torch\nimport torch.nn as nn\n\ntorch.set_num_threads(1)                                   # single-threaded CPU, as in the paper\nc, h, w = 192, 28, 28                                      # a common feature-map size in the second stage\ncp = c // 4                                                # PConv convolves only 1/4 of the channels\n\nclass PConv(nn.Module):\n    def __init__(self, c, cp):\n        super().__init__()\n        self.cp = cp\n        self.conv = nn.Conv2d(cp, cp, 3, padding=1, bias=False)\n\n    def forward(self, x):\n        x1, x2 = torch.split(x, [self.cp, x.shape[1] - self.cp], dim=1)\n        return torch.cat((self.conv(x1), x2), 1)           # the other 3/4 of the channels pass through untouched\n\n# (operator, input channels, FLOPs, memory access); FLOPs counted as multiply-adds (MACs), memory access as elements of feature maps read and written plus weights read\nops = {\n    \"Conv 3×3\": (nn.Conv2d(c, c, 3, padding=1, bias=False), c, h * w * 9 * c * c, h * w * 2 * c + 9 * c * c),\n    \"DWConv 3×3\": (nn.Conv2d(c, c, 3, padding=1, groups=c, bias=False), c, h * w * 9 * c, h * w * 2 * c + 9 * c),\n    \"PConv 3×3\": (PConv(c, cp), c, h * w * 9 * cp * cp, h * w * 2 * cp + 9 * cp * cp),\n    \"DWConv ×6\": (nn.Conv2d(6 * c, 6 * c, 3, padding=1, groups=6 * c, bias=False), 6 * c,\n                  h * w * 9 * 6 * c, h * w * 2 * 6 * c + 9 * 6 * c),\n}\nwith torch.no_grad():\n    for name, (op, cin, flops, mem) in ops.items():\n        x = torch.randn(1, cin, h, w)\n        net = nn.Sequential(*[op] * 10)                    # the same layer stacked 10 times\n        for _ in range(20):\n            net(x)                                         # warm-up\n        t = time.perf_counter()\n        for _ in range(200):\n            net(x)\n        ms = (time.perf_counter() - t) / 200 * 1000\n        total = flops * 10\n        print(f\"{name:<10}  FLOPs {total / 1e6:6.1f}M  MACs per access {flops / mem:5.1f}  \"\n              f\"latency {ms:5.2f} ms  FLOPS {total / ms / 1e6:4.1f} G/s\")\n```\n\n```\nConv 3×3    FLOPs 2601.1M  MACs per access 411.0  latency 32.83 ms  FLOPS 79.2 G/s\nDWConv 3×3  FLOPs   13.5M  MACs per access   4.5  latency  0.83 ms  FLOPS 16.4 G/s\nPConv 3×3   FLOPs  162.6M  MACs per access 169.3  latency  2.72 ms  FLOPS 59.8 G/s\nDWConv ×6   FLOPs   81.3M  MACs per access   4.5  latency 10.29 ms  FLOPS  7.9 G/s\n```\n\nThese results come from an Intel i9-13900K. On another machine the numbers will differ, though the trend should be about the same.\n\nStart with the FLOPS column. Ordinary convolution does 411 multiply-adds for every number it accesses, making the fullest use of the hardware; depthwise convolution does only 4.5, and its FLOPS drops to about a fifth. PConv does 169 per number, with FLOPS about three-quarters of ordinary convolution's.\n\nTo be fair, with the same 192 channels and the same 10-layer stack, depthwise convolution without widening is actually the fastest, at just 0.83 ms, because its FLOPs are just so few. But it's never used alone like that. With the channels expanded 6×, its FLOPs are only half of PConv's, yet its latency is nearly 4 times PConv's.\n\nThe paper ran the same experiment on a 2080Ti, an i9-9900X, and a Cortex-A72. Averaged over four feature-map sizes, PConv has 1/16 the FLOPs of ordinary convolution, and its FLOPS is 10.5× (GPU), 6.2× (CPU), and 22.8× (ARM) that of depthwise convolution.\n\n### Followed by a 1×1 Convolution\n\nPConv looks at only a quarter of the channels, so the rest of the information has to come from the 1×1 convolution that follows. Together, their receptive field looks like a T. The center position sees all the channels, while the eight surrounding positions see only $c_p$ channels.\n\nThe authors used a pretrained ResNet-18 to check whether this shape makes sense. For each 3×3 kernel, they computed the weight magnitude at each of the nine positions and noted which one was largest. In all four stages, the position that was most often the largest was the center. The center matters more than its surroundings to begin with, so spending a bit more computation on it is reasonable.\n\nThen why not just use a T-shaped convolution? Because splitting it is cheaper. A T-shaped convolution has $h\\times w\\times(k^2 c_p c + c(c-c_p))$ FLOPs, while PConv plus a 1×1 convolution has $h\\times w\\times(k^2 c_p^2 + c^2)$. With $c_p = c/4$, the former is $3c^2$ and the latter only $1.5625c^2$.\n\nThe authors also checked whether this combination can stand in for ordinary convolution. They took the inputs and outputs of the first 3×3 convolution in each stage of a pretrained ResNet-50 and trained small networks to imitate it. Averaged over the four stages, the test loss is 0.0148 for depthwise convolution plus 1×1, 0.0133 for grouped convolution plus 1×1, and 0.0123 for PConv plus 1×1, which comes closest to the original ordinary convolution.\n\n### FasterNet\n\n[![The top half shows FasterNet's overall architecture: the input passes through an Embedding layer, then four stages, each made of several FasterNet Blocks, with Merging layers between stages; the feature map shrinks from h/4×w/4 to h/32×w/32, followed at the end by global pooling, a 1×1 convolution, and a fully connected layer that produces the output. The bottom left zooms in on partial convolution, where only the first c_p channels of the input are multiplied with c_p k×k kernels and the other channels go straight to the output along an Identity arrow; on the right is the FasterNet Block, made up in order of a PConv 3×3, a 1×1 convolution, BN and ReLU, another 1×1 convolution, and finally an addition with the input](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)](/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png)\n\n*Figure 4 from the paper, excerpted from [**page 5 of the v3 PDF**](https://arxiv.org/pdf/2303.03667v3#page=5), CC BY 4.0.*\n\nFasterNet's skeleton is very plain. The stem is a 4×4 convolution with stride 4, and between stages a 2×2 convolution with stride 2 does the downsampling. Each block is a PConv followed by two 1×1 convolutions that expand the channels 2× in between, with a residual added at the end.\n\nTwo details are worth mentioning. First, normalization and activation come only after the middle layer, so the whole block has just one set; the authors believe that too many would reduce feature diversity and also slow things down. Second, the normalization is BatchNorm, which can be merged into the adjacent convolution at inference.\n\nThe last two stages have less memory access and higher FLOPS, so the authors put more of the blocks and computation toward the back; T0's four stages, for example, have 1, 2, 8, and 2 blocks. From small to large, the family is T0, T1, T2, S, M, and L, with GELU for the small models and ReLU for the large ones.\n\n## Discussion\n\n### ImageNet\n\nThe authors measured speed on three devices. The GPU is a 2080Ti, measuring throughput at batch size 32; the CPU is an i9-9900X and the ARM chip a Cortex-A72, both single-threaded and measuring latency at batch size 1. Here are a few groups of models with similar accuracy:\n\n| Model | FLOPs | GPU (images/s) | CPU (ms) | ARM (ms) | Top-1 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| MobileViT-XXS | 0.42G | 2,393 | 30.8 | 348 | 69.0% |\n| MobileNetV2 | 0.31G | 4,198 | 12.2 | 442 | 72.0% |\n| ShuffleNetV2 ×1.5 | 0.30G | 4,878 | 12.1 | 266 | **72.6%** |\n| **FasterNet-T0** | 0.34G | **6,807** | **9.2** | **143** | 71.9% |\n| ResNet-50 | 4.11G | 959 | 73.0 | 1,131 | 78.8% |\n| **FasterNet-T2** | 1.91G | **1,991** | **33.5** | **497** | **78.9%** |\n| Swin-T | 4.51G | 609 | 122.2 | 1,424 | 81.3% |\n| ConvNeXt-T | 4.47G | 657 | 86.3 | 1,889 | **82.1%** |\n| **FasterNet-S** | 4.56G | **1,029** | **71.2** | **1,103** | 81.3% |\n| Swin-B | 15.47G | 237 | 349.2 | OOM | 83.5% |\n| ConvNeXt-B | 15.38G | 253 | 257.1 | OOM | **83.8%** |\n| **FasterNet-L** | 15.52G | **323** | **219.5** | OOM | 83.5% |\n\n*Table: Excerpted from Table 3 of the paper. MobileViT's input is 256×256 and the rest 224×224; OOM means out of memory.*\n\nIn every group, FasterNet is the fastest on all three devices. The most striking is the last group. FasterNet-L, ConvNeXt-B, and Swin-B have nearly the same FLOPs, all around 15.4G to 15.5G, yet FasterNet-L's GPU throughput is 28% to 36% higher and its CPU time 15% to 37% lower. Same FLOPs, different speeds, and that's exactly what the whole paper sets out to show.\n\nThe accuracy needs a closer look. The abstract says T0 is 2.9 percentage points more accurate than MobileViT-XXS, which is true, but MobileNetV2 and ShuffleNetV2 in the same group are both more accurate than T0. FasterNet-S ties Swin-T at 81.3%, 0.8 below ConvNeXt-T. The paper calls FasterNet-L “comparable” to ConvNeXt-B, but it's actually 0.3 lower. What FasterNet buys is speed, with accuracy roughly level and sometimes a little behind.\n\n### Detection and Segmentation\n\nOn COCO with Mask R-CNN trained for 12 epochs, FasterNet-S reaches 39.9 box AP against ResNet-50's 38.0, with nearly identical GPU latency (54.3 ms versus 54.9 ms). FasterNet-L gets 44.0 against PVT-Large's 42.9, while latency drops from 152.2 ms to 93.8 ms.\n\n### Ablation\n\n| Setting | GPU (images/s) | CPU (ms) | ARM (ms) | Top-1 |\n| --- | ---: | ---: | ---: | ---: |\n| T0, $c_p = c/4$ (default) | 6,807 | 9.2 | 143 | 71.9% |\n| T0\\*, $c_p = c/2$ | 6,626 | 9.6 | 145 | 71.7% |\n| T0\\*, $c_p = c/8$ | 6,204 | 8.9 | 140 | 71.3% |\n| T0, LayerNorm instead of BatchNorm | 5,515 | 10.7 | 159 | 71.9% |\n| T0, ReLU instead of GELU | 6,929 | 8.2 | 114 | 71.3% |\n| T2, GELU instead of ReLU | 1,985 | 35.4 | 557 | 78.7% |\n\n*Table: Excerpted from Table 5 of the paper. T0\\* is a version with width and depth adjusted to keep the complexity similar; T2 uses ReLU by default, so the comparison here is a switch to GELU.*\n\nA $c_p$ of 1/4 strikes the best balance. Too large and it degenerates into ordinary convolution; too small and it can't capture spatial features. LayerNorm and BatchNorm give the same accuracy, but LayerNorm can't be merged into the convolution, and GPU throughput drops by a fifth. As for activation functions, GELU is more accurate for small models and ReLU works better for large ones; the authors guess that small models need GELU's nonlinearity more.\n\nIn the appendix, the authors also list two limitations. PConv's stride can only be 1, since otherwise the convolved channels and the untouched channels won't line up, so downsampling has to be left to other layers. And FasterNet is built entirely from small convolutions with a limited receptive field; the authors leave enlarging the receptive field to future work.\n\n## Conclusion\n\nLatency is FLOPs divided by FLOPS, and that equation is the thing most worth remembering from this paper. Focus only on shrinking the numerator, and if the denominator falls faster, the network ends up slower. Depthwise convolution is exactly that case, saving on computation but spending the time moving memory around.\n\nPConv is so simple it's almost embarrassing. Convolve a quarter of the channels and stitch them back on, that's all, and a few lines are enough to bring it into your own network.\n\nFinally, always measure speed on your own device. The paper used a 2080Ti, an i9-9900X, and a Cortex-A72; on an i9-13900K we saw the same trend, but quite different numbers. Move to a phone NPU or a different inference framework, and which one is faster could flip entirely, so don't draw conclusions straight from the paper's tables.\n","imageSizes":{"/content/papers/assets/20f1c9a908d887f3d0a9df04a50fcc019d38cf455e2b89451d3b2c8469b91f94.png":{"width":2083,"height":650},"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png":{"width":1012,"height":500},"/content/papers/assets/25dba5e2fc3ece26c7c5fd70741eb94aaf914907a40406aabfb39756ab3f1144.png":{"width":1767,"height":742}}}