{"notes":[{"id":"lightweight/lsnet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"LSNet","subtitle":"See Wide, Focus Close","author":"Z. Yuan","paperDate":"2025-03-29","publishedOn":"2026-03-09","paperTitle":"LSNet: See Large, Focus Small","paperUrl":"https://arxiv.org/abs/2503.23135","bibliography":{"title":"LSNet: See Large, Focus Small","authors":["Ao Wang","Hui Chen","Zijia Lin","Jungong Han","Guiguang Ding"],"url":"https://arxiv.org/abs/2503.23135","pdfUrl":"https://arxiv.org/pdf/2503.23135v1","codeUrl":"https://github.com/jameslahm/lsnet","preprint":{"id":"2503.23135","submittedOn":"2025-03-29","url":"https://arxiv.org/abs/2503.23135"},"publication":{"venue":"CVPR","year":2025,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2025/html/Wang_LSNet_See_Large_Focus_Small_CVPR_2025_paper.html"},"citation":"@misc{lightweightlsnet2025,\n  title = {{LSNet: See Large, Focus Small}},\n  author = {Ao Wang and Hui Chen and Zijia Lin and Jungong Han and Guiguang Ding},\n  year = {2025},\n  eprint = {2503.23135},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2503.23135}\n}\n","citationPath":"/content/papers/citations/lightweight/lsnet.d31e2f81ec32.bib"},"thumbnail":{"src":"/content/papers/assets/6d3e348c9f8512697f7230b2ac3fde0efdd4fd84198dbb519d931acd26bcbb59.png","width":1728,"height":948},"locale":"en","path":"/content/papers/en/lightweight/lsnet.663b49580f34.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"perception-and-aggregation","text":"Perception and Aggregation","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"ls-convolution","text":"LS Convolution","depth":3},{"id":"building-the-network","text":"Building the Network","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"ablation","text":"Ablation","depth":3},{"id":"deployment","text":"Deployment","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/repvit","relation":"referenced","topic":""},{"id":"object-detection/yolov10","relation":"referenced","topic":""},{"id":"classic-cnns/overlock","relation":"referenced","topic":""}]},{"id":"reparameterization/repvit","category":"reparameterization","legacyPath":"reparameterization/2307-repvit","legacySlug":"2307-repvit","legacyPaths":[],"title":"RepViT","subtitle":"Revisiting Mobile CNN from the ViT Perspective","author":"Z. Yuan","paperDate":"2023-07-18","publishedOn":"2024-07-19","paperTitle":"RepViT: Revisiting Mobile CNN From ViT Perspective","paperUrl":"https://arxiv.org/abs/2307.09283","bibliography":{"title":"RepViT: Revisiting Mobile CNN From ViT Perspective","authors":["Ao Wang","Hui Chen","Zijia Lin","Jungong Han","Guiguang Ding"],"url":"https://arxiv.org/abs/2307.09283","pdfUrl":"https://arxiv.org/pdf/2307.09283","codeUrl":null,"preprint":{"id":"2307.09283","submittedOn":"2023-07-18","url":"https://arxiv.org/abs/2307.09283"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://arxiv.org/abs/2307.09283"},"citation":"@misc{reparameterizationrepvit2023,\n  title = {{RepViT: Revisiting Mobile CNN From ViT Perspective}},\n  author = {Ao Wang and Hui Chen and Zijia Lin and Jungong Han and Guiguang Ding},\n  year = {2023},\n  eprint = {2307.09283},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2307.09283}\n}\n","citationPath":"/content/papers/citations/reparameterization/repvit.583fd659fe6d.bib"},"thumbnail":{"src":"/content/papers/assets/f501c7d5eb209527b5401ba7607cd34496d2dda89aeb1848aa9927e0d0f713eb.jpg","width":1224,"height":704},"locale":"en","path":"/content/papers/en/reparameterization/repvit.8b88345cbf24.md","headings":[{"id":"problem-definition","text":"Problem Definition","depth":2},{"id":"solution-approach","text":"Solution Approach","depth":2},{"id":"redesigning-the-architecture","text":"Redesigning the Architecture","depth":3},{"id":"reparameterizing-the-structure","text":"Reparameterizing the Structure","depth":3},{"id":"reducing-expansion-ratio-and-increasing-width","text":"Reducing Expansion Ratio and Increasing Width","depth":3},{"id":"adjusting-the-stem-structure","text":"Adjusting the Stem Structure","depth":3},{"id":"adjusting-the-downsampling-structure","text":"Adjusting the Downsampling Structure","depth":3},{"id":"simple-classifier","text":"Simple Classifier","depth":3},{"id":"adjusting-stage-ratios","text":"Adjusting Stage Ratios","depth":3},{"id":"kernel-size-selection","text":"Kernel Size Selection","depth":3},{"id":"squeeze-and-excitation-se-layer-decision","text":"Squeeze-and-Excitation (SE) Layer Decision","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"performance-on-imagenet-1k","text":"Performance on ImageNet-1K","depth":3},{"id":"ablation-study---reparameterized-structure","text":"Ablation Study - Reparameterized Structure","depth":3},{"id":"ablation-study---impact-of-se-layers","text":"Ablation Study - Impact of SE Layers","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/fastvit","relation":"comparison","topic":"Fast visual backbones for deployment"},{"id":"reparameterization/mobileone","relation":"comparison","topic":"Fast visual backbones for deployment"}]},{"id":"object-detection/yolov10","category":"object-detection","legacyPath":"object-detection/2405-yolov10","legacySlug":"2405-yolov10","legacyPaths":[],"title":"YOLOv10","subtitle":"Goodbye NMS","author":"Z. Yuan","paperDate":"2024-05-23","publishedOn":"2025-07-19","paperTitle":"YOLOv10: Real-Time End-to-End Object Detection","paperUrl":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/c34ddd05eb089991f06f3c5dc36836e0-Abstract-Conference.html","bibliography":{"title":"YOLOv10: Real-Time End-to-End Object Detection","authors":["Ao Wang","Hui Chen","Lihao Liu","Kai Chen","Zijia Lin","Jungong Han","Guiguang Ding"],"url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/c34ddd05eb089991f06f3c5dc36836e0-Abstract-Conference.html","pdfUrl":"https://proceedings.neurips.cc/paper_files/paper/2024/file/c34ddd05eb089991f06f3c5dc36836e0-Paper-Conference.pdf","codeUrl":"https://github.com/THU-MIG/yolov10","preprint":{"id":"2405.14458","submittedOn":"2024-05-23","url":"https://arxiv.org/abs/2405.14458"},"publication":{"venue":"NeurIPS","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/c34ddd05eb089991f06f3c5dc36836e0-Abstract-Conference.html"},"citation":"@misc{objectdetectionyolov102024,\n  title = {{YOLOv10: Real-Time End-to-End Object Detection}},\n  author = {Ao Wang and Hui Chen and Lihao Liu and Kai Chen and Zijia Lin and Jungong Han and Guiguang Ding},\n  year = {2024},\n  eprint = {2405.14458},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2405.14458}\n}\n","citationPath":"/content/papers/citations/object-detection/yolov10.f624ec9e20b6.bib"},"thumbnail":{"src":"/content/papers/assets/43843817d1e6667933eb36b7b588db3ca318fe74e00d2d47095426923f3a5b44.jpg","width":1614,"height":536},"locale":"en","path":"/content/papers/en/object-detection/yolov10.0dbaf7fa020d.md","headings":[{"id":"problem-definition","text":"Problem Definition","depth":2},{"id":"solution","text":"Solution","depth":2},{"id":"dual-annotation-architecture","text":"Dual Annotation Architecture","depth":3},{"id":"efficiency-and-accuracy-optimization","text":"Efficiency and Accuracy Optimization","depth":3},{"id":"accuracy-oriented-enhancements","text":"Accuracy-Oriented Enhancements","depth":3},{"id":"implementation-details","text":"Implementation Details","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"comparison-with-sota-methods","text":"Comparison with SOTA Methods","depth":3},{"id":"ablation-study","text":"Ablation Study","depth":3},{"id":"can-nms-free-truly-replace-traditional-methods","text":"Can NMS-free Truly Replace Traditional Methods?","depth":3},{"id":"why-doesnt-simplification-drag-performance","text":"Why Doesn’t Simplification Drag Performance?","depth":3},{"id":"accuracy-improvements-with-nearly-no-latency-increase","text":"Accuracy Improvements with Nearly No Latency Increase?","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{"experimental-results-and-discussion":"discussion"},"guide":null,"relatedReading":[{"id":"object-detection/yolov11","relation":"comparison","topic":"Recent designs for efficient real-time detection"},{"id":"object-detection/yolo-tiny","relation":"comparison","topic":"Recent designs for efficient real-time detection"}]},{"id":"classic-cnns/overlock","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"OverLoCK","subtitle":"See the Big Picture First","author":"Z. Yuan","paperDate":"2025-02-27","publishedOn":"2026-02-24","paperTitle":"OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels","paperUrl":"https://arxiv.org/abs/2502.20087","bibliography":{"title":"OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels","authors":["Meng Lou","Yizhou Yu"],"url":"https://arxiv.org/abs/2502.20087","pdfUrl":"https://arxiv.org/pdf/2502.20087v3","codeUrl":"https://github.com/LMMMEng/OverLoCK","preprint":{"id":"2502.20087","submittedOn":"2025-02-27","url":"https://arxiv.org/abs/2502.20087"},"publication":{"venue":"CVPR","year":2025,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2025/html/Lou_OverLoCK_An_Overview-first-Look-Closely-next_ConvNet_with_Context-Mixing_Dynamic_Kernels_CVPR_2025_paper.html"},"citation":"@misc{classiccnnsoverlock2025,\n  title = {{OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels}},\n  author = {Meng Lou and Yizhou Yu},\n  year = {2025},\n  eprint = {2502.20087},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.20087}\n}\n","citationPath":"/content/papers/citations/classic-cnns/overlock.a1eecba8875b.bib"},"thumbnail":{"src":"/content/papers/assets/99458c439c8f5458671d09ec6d68f52f31a20702b6a0bdba99e2fe33dbf86ea7.png","width":2112,"height":681},"locale":"en","path":"/content/papers/en/classic-cnns/overlock.4a63c66446e8.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"intermediate-layers-need-direction","text":"Intermediate Layers Need Direction","depth":3},{"id":"kernels-cannot-see-far-enough","text":"Kernels Cannot See Far Enough","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"split-into-three-subnetworks","text":"Split into Three Subnetworks","depth":3},{"id":"contmix-put-the-whole-image-into-the-kernel","text":"ContMix: Put the Whole Image into the Kernel","depth":3},{"id":"four-sizes","text":"Four Sizes","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"add-one-thing-at-a-time","text":"Add One Thing at a Time","depth":3},{"id":"against-other-dynamic-operators","text":"Against Other Dynamic Operators","depth":3},{"id":"using-it-in-practice","text":"Using It in Practice","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/internimage","relation":"referenced","topic":""},{"id":"classic-cnns/dcnv4","relation":"referenced","topic":""}]}],"text":"Spatial mixing in lightweight networks usually offers two choices. Self-attention sees widely but costs more; small kernels are cheap but see narrowly, and their weights stay fixed after training.\n\nThis paper comes from Tsinghua University. First author Ao Wang is also the first author of RepViT and YOLOv10. The inspiration comes from the retina's two kinds of photoreceptors. Rods are spread around the periphery, seeing broadly but coarsely; cones cluster in the center, seeing a smaller region in sharp detail. The title, See Large, Focus Small, describes this division of labor.\n\n\n> [!TIP]\n> \n> Two other works from the same first author:\n> \n> - **[[23.07] RepViT: Revisiting Mobile CNN from the ViT Perspective](/blog/papers/reparameterization/repvit)**\n> - **[[24.05] YOLOv10: Goodbye NMS](/blog/papers/object-detection/yolov10)**\n\n\n## Defining the Problem\n\n### Perception and Aggregation\n\nThe authors divide token mixing—the exchange of spatial information—into two steps.\n\n- **Perception**: determine which surrounding locations relate to the current one, and how strongly, by computing aggregation weights.\n- **Aggregation**: use those weights to sum surrounding features.\n\n[![Figure 1 from the paper: (a) self-attention begins with a grid over a cat-and-dog photo, with a dark-blue query cell and orange perception arrows. Softmax produces weights over a wide region, with useful weights in green and redundant weights in red; blue aggregation arrows cover the same region as perception. (b) convolution uses a small 3×3 region for both perception and aggregation, with a fixed kernel. (c) the human retina has peripheral rods and central cones. Orange arrows enter from a wide blurry image, labeled See Large, Perception; blue arrows focus on a sharp local region, labeled Focus Small, Aggregation. (d) a line plot has eccentricity from the fovea horizontally. Blue rod density is high in the periphery and zero centrally; red cone density has a sharp central peak.](/content/papers/assets/85567a02dcfb2f88ff1be569a4d077a422f781df88b73b63effe301953d2985b.png)](/content/papers/assets/85567a02dcfb2f88ff1be569a4d077a422f781df88b73b63effe301953d2985b.png)\n\n*Figure 1 from Wang et al., taken from [**PDF page 2**](https://arxiv.org/pdf/2503.23135v1#page=2), CC BY 4.0. The original caption is omitted.*\n\nThrough this lens, both common token mixers have a problem.\n\nSelf-attention uses the same wide region for perception and aggregation. It compares a query with all positions, applies softmax, then forms a weighted sum over them all. A wider region costs more compute and often assigns many weights to uninformative areas such as background—the red cells in panel (a).\n\nConvolution also uses the same region for perception and aggregation, only a small one. Aggregation weights are the kernel, based solely on relative position, shared across locations, and fixed after training. Lightweight networks usually have small kernels, limiting their view and preventing content-dependent adjustment.\n\nBoth aggregate over the region they see. The authors want to separate the two extents: wide perception, small-region aggregation.\n\n## Solving the Problem\n\n### LS Convolution\n\n[![Figure 3 from the paper: (a) LS convolution. The upper orange dashed Large Kernel Perception box takes H×W×C input through a 1×1 convolution to C/2, a K_L×K_L depthwise convolution, and two 1×1 convolutions, giving H×W×D, reshaped to H×W×G×K_s×K_s. The lower blue dashed Small Kernel Aggregation box assigns a K_s×K_s kernel to each location, shared by C/G channels in a group, to convolve a 3×3 neighborhood. (b) the LSNet architecture runs upward through Stem, three LS Block stages and Downsample, a final MSA Block stage, Pooling, and FC. The expanded Stem has three stride-2 3×3 convolutions; Downsample has stride-2 depthwise 3×3 plus 1×1. LS Block contains DW3×3, SE, FFN, LS Convolution, FFN; MSA Block replaces LS Convolution with MHSA.](/content/papers/assets/6d3e348c9f8512697f7230b2ac3fde0efdd4fd84198dbb519d931acd26bcbb59.png)](/content/papers/assets/6d3e348c9f8512697f7230b2ac3fde0efdd4fd84198dbb519d931acd26bcbb59.png)\n\n*Figure 3 from Wang et al., taken from [**PDF page 5**](https://arxiv.org/pdf/2503.23135v1#page=5), CC BY 4.0. The original caption is omitted.*\n\nLS (Large-Small) convolution implements the idea in two parts:\n\n1. **Large-kernel perception**: a 1×1 convolution halves input channels, a $K_L\\times K_L$ depthwise convolution sees a broad neighborhood, and 1×1 convolutions produce weights for every location. The default is $K_L=7$.\n2. **Small-kernel aggregation**: arrange the weights at each location into $G$ kernels of size $K_S\\times K_S$, then form a weighted sum over a small local neighborhood. The default is $K_S=3$, sharing one kernel among every 8 channels in a group.\n\nAggregation weights are dynamic and location-specific, decided after seeing a 7×7 region, while the values actually summed come from just 3×3 neighbors. Write it out and use gradients to see which inputs affect the center output:\n\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ntorch.manual_seed(0)\nC, H, W = 16, 15, 15\nG, KL, KS = 2, 7, 3                                   # 2 groups; 7×7 for perception, 3×3 for aggregation\npw1 = nn.Conv2d(C, C // 2, 1)\ndw = nn.Conv2d(C // 2, C // 2, KL, padding=KL // 2, groups=C // 2)\npw2 = nn.Conv2d(C // 2, G * KS * KS, 1)\n\ndef ls_conv(x, detach_weights=False):\n    w = pw2(dw(pw1(x)))                               # Large-kernel perception: a 3×3 kernel for each location and group\n    if detach_weights:\n        w = w.detach()                                # Keep only the path through values being summed\n    w = w.view(G, 1, KS * KS, H * W)\n    patches = F.unfold(x, KS, padding=KS // 2).view(G, C // G, KS * KS, H * W)\n    return (patches * w).sum(2).view(1, C, H, W)      # Small-kernel aggregation: channels in one group share a kernel\n\ndef footprint(detach_weights):\n    x = torch.randn(1, C, H, W, requires_grad=True)\n    ls_conv(x, detach_weights)[0, :, H // 2, W // 2].sum().backward()\n    grid = x.grad[0].abs().sum(0) > 0\n    rows, cols = grid.nonzero().T\n    return f\"{int(grid.sum())} positions, spanning {int(rows.max() - rows.min() + 1)}×{int(cols.max() - cols.min() + 1)}\"\n\nprint(\"Inputs affecting the center output:\", footprint(False))\nprint(\"Values actually summed:\", footprint(True))\n```\n\n```text\nInputs affecting the center output: 49 positions, spanning 7×7\nValues actually summed: 9 positions, spanning 3×3\n```\n\nForty-nine positions help determine weights; only 9 contribute values to the sum. That is See Large, Focus Small. The actual module also includes normalization and activation functions, omitted here.\n\nThis keeps compute low. The wide-region component uses only depthwise convolution, while the small-region component sums only 9 values. Overall cost scales linearly with resolution, rather than quadratically with the region size as in self-attention.\n\n\n> [!TIP]\n> \n> Compare LS convolution's weight generation with OverLoCK, which also sees a wide region before deciding local weights:\n> \n> - **[[25.02] OverLoCK: See the Big Picture First](/blog/papers/classic-cnns/overlock)**\n\n\n### Building the Network\n\nAn LS block uses LS convolution for spatial mixing with a residual connection, adds a depthwise convolution and SE layer, and uses FFNs for channel mixing.\n\nLSNet has four stages. The first three use LS blocks. At the last stage, resolution is just $H/64$, and MSA blocks use multi-head self-attention. LSNet is therefore a hybrid, with attention in its final stage.\n\nThe three sizes, T, S, and B, take 0.3G, 0.5G, and 1.3G FLOPs. Blocks are concentrated in later stages because early, high-resolution stages cost more time.\n\n## Discussion\n\n### ImageNet\n\nThroughput is measured on one RTX 3090, with batch size set to the largest power of two that fits in memory:\n\n| Model | Parameters | FLOPs | Throughput (images/s) | top-1 |\n| --- | ---: | ---: | ---: | ---: |\n| FasterNet-T0 | 3.9M | 0.3G | 14467 | 71.9% |\n| EfficientViT-M3 | 6.9M | 0.3G | 14613 | 73.4% |\n| StarNet-S1 | 2.9M | 0.4G | 5034 | 73.5% |\n| LSNet-T | 11.4M | 0.3G | **14708** | **74.9%** |\n| UniRepLKNet-A | 4.4M | 0.6G | 3931 | 77.0% |\n| RepViT-M0.9 | 5.1M | 0.8G | 4817 | 77.4% |\n| SHViT-S3 | 14.2M | 0.6G | 8993 | 77.4% |\n| LSNet-S | 16.1M | 0.5G | **9023** | **77.8%** |\n| FastViT-T12 | 6.8M | 1.4G | 2586 | 79.1% |\n| RepViT-M1.1 | 8.2M | 1.3G | 3604 | 79.4% |\n| LSNet-B | 23.2M | 1.3G | **3996** | **80.3%** |\n\n*Table: selected rows from the paper's Table 1, excluding distilled variants.*\n\nAt similar FLOPs, LSNet ranks highly in both accuracy and throughput. LSNet-T beats StarNet-S1 by 1.4 percentage points with nearly three times the throughput; LSNet-B beats RepViT-M1.1 by 0.9 points. Distillation from RegNetY-16GF adds another 1.2 to 1.3 points across the three sizes.\n\nTwo things need to be considered alongside this.\n\nFirst, parameters. LSNet-T takes only 0.3G FLOPs but has 11.4M parameters, four times StarNet-S1's count. LSNet-B has 23.2M. If model size or memory limits deployment, the ranking needs another look.\n\nSecond, the measurement protocol. The paper reports only large-batch throughput on an RTX 3090, with no phone or CPU latency. StarNet's strength is batch-1 latency on an iPhone. The two papers measure different things; the nearly threefold speedup in this table cannot simply be transferred to phones.\n\n### Ablation\n\nThese experiments train LSNet-T for only 100 epochs, with a 71.6% baseline:\n\n| Change | top-1 |\n| --- | ---: |\n| LS convolution (baseline) | 71.6% |\n| Remove LS convolution | 69.3% |\n| Remove large-kernel perception | 70.5% |\n| Perception kernel from 7 to 3 | 70.9% |\n| Perception kernel from 7 to 9 | 71.5% |\n| Remove small-kernel aggregation (large kernel followed by ordinary small kernel) | 70.1% |\n| Aggregation kernel from 3 to 1 | 69.6% |\n| Aggregation kernel from 3 to 5 | 71.6% |\n\n*Table: from the paper's Tables 5 and 7.*\n\nTwo rows deserve a closer look.\n\n“Remove small-kernel aggregation” feeds the result of large-kernel perception directly into an ordinary static small kernel—the most intuitive way to combine large and small kernels. It reaches only 70.1%, 1.5 points below LS convolution. Using wide-region information to decide weights helps more than simply stacking the kernels.\n\nIncreasing aggregation size from 3 to 5 leaves accuracy unchanged while raising FLOPs from 0.31G to 0.34G. Increasing perception from 7 to 9 brings no improvement either. Seeing 7×7 and summing 3×3 is enough.\n\nThe authors also swap LS convolution into other networks. Replacing every ResNet-50 3×3 convolution cuts FLOPs from 4.1G to 2.6G and raises accuracy from 78.8% to 80.7%. Replacing DeiT-T's self-attention cuts FLOPs from 1.3G to 0.9G and raises accuracy from 72.2% to 73.0%.\n\n### Deployment\n\nSmall-kernel aggregation in LS convolution needs location-specific kernels, beyond PyTorch's built-in `conv2d`. The official code uses a dedicated GPU operator written in Triton. A GitHub user [asks why built-in convolution is not used](https://github.com/jameslahm/lsnet/issues/8) and mentions poor TensorRT conversion compatibility. Deployment to phones or other inference engines requires adapting this step, for example with `unfold` and elementwise multiplication, as in the demonstration above.\n\n## Conclusion\n\nLSNet separates token mixing into perception and aggregation, giving each a different extent. A 7×7 depthwise convolution sees the neighborhood and decides each location's 3×3 weights; aggregation then sums just 9 weighted values. The ablations clearly show that wide-region information is more useful for determining weights than for simply stacking large and small kernels.\n\nLS convolution itself is more worth taking away than the whole LSNet. It needs only depthwise convolution and location-wise weighted sums. Swapped into ResNet-50, it cuts FLOPs by nearly 40% while adding 1.9 percentage points of accuracy.\n","imageSizes":{"/content/papers/assets/85567a02dcfb2f88ff1be569a4d077a422f781df88b73b63effe301953d2985b.png":{"width":1743,"height":432},"/content/papers/assets/6d3e348c9f8512697f7230b2ac3fde0efdd4fd84198dbb519d931acd26bcbb59.png":{"width":1728,"height":948}}}