{"notes":[{"id":"classic-cnns/overlock","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"OverLoCK","subtitle":"See the Big Picture First","author":"Z. Yuan","paperDate":"2025-02-27","publishedOn":"2026-02-24","paperTitle":"OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels","paperUrl":"https://arxiv.org/abs/2502.20087","bibliography":{"title":"OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels","authors":["Meng Lou","Yizhou Yu"],"url":"https://arxiv.org/abs/2502.20087","pdfUrl":"https://arxiv.org/pdf/2502.20087v3","codeUrl":"https://github.com/LMMMEng/OverLoCK","preprint":{"id":"2502.20087","submittedOn":"2025-02-27","url":"https://arxiv.org/abs/2502.20087"},"publication":{"venue":"CVPR","year":2025,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2025/html/Lou_OverLoCK_An_Overview-first-Look-Closely-next_ConvNet_with_Context-Mixing_Dynamic_Kernels_CVPR_2025_paper.html"},"citation":"@misc{classiccnnsoverlock2025,\n  title = {{OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels}},\n  author = {Meng Lou and Yizhou Yu},\n  year = {2025},\n  eprint = {2502.20087},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.20087}\n}\n","citationPath":"/content/papers/citations/classic-cnns/overlock.a1eecba8875b.bib"},"thumbnail":{"src":"/content/papers/assets/99458c439c8f5458671d09ec6d68f52f31a20702b6a0bdba99e2fe33dbf86ea7.png","width":2112,"height":681},"locale":"en","path":"/content/papers/en/classic-cnns/overlock.4a63c66446e8.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"intermediate-layers-need-direction","text":"Intermediate Layers Need Direction","depth":3},{"id":"kernels-cannot-see-far-enough","text":"Kernels Cannot See Far Enough","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"split-into-three-subnetworks","text":"Split into Three Subnetworks","depth":3},{"id":"contmix-put-the-whole-image-into-the-kernel","text":"ContMix: Put the Whole Image into the Kernel","depth":3},{"id":"four-sizes","text":"Four Sizes","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"add-one-thing-at-a-time","text":"Add One Thing at a Time","depth":3},{"id":"against-other-dynamic-operators","text":"Against Other Dynamic Operators","depth":3},{"id":"using-it-in-practice","text":"Using It in Practice","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/internimage","relation":"referenced","topic":""},{"id":"classic-cnns/dcnv4","relation":"referenced","topic":""}]},{"id":"classic-cnns/internimage","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"InternImage","subtitle":"Miracles by Brute Force?","author":"Z. Yuan","paperDate":"2022-11-10","publishedOn":"2025-09-04","world":"external/internimage","paperTitle":"InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions","paperUrl":"https://arxiv.org/abs/2211.05778","bibliography":{"title":"InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions","authors":["Wenhai Wang","Jifeng Dai","Zhe Chen","Zhenhang Huang","Zhiqi Li","Xizhou Zhu","Xiaowei Hu","Tong Lu","Lewei Lu","Hongsheng Li","Xiaogang Wang","Yu Qiao"],"url":"https://arxiv.org/abs/2211.05778","pdfUrl":"https://arxiv.org/pdf/2211.05778v4","codeUrl":null,"preprint":{"id":"2211.05778","submittedOn":"2022-11-10","url":"https://arxiv.org/abs/2211.05778"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Wang_InternImage_Exploring_Large-Scale_Vision_Foundation_Models_With_Deformable_Convolutions_CVPR_2023_paper.html"},"citation":"@misc{classiccnnsinternimage2022,\n  title = {{InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions}},\n  author = {Wenhai Wang and Jifeng Dai and Zhe Chen and Zhenhang Huang and Zhiqi Li and Xizhou Zhu and Xiaowei Hu and Tong Lu and Lewei Lu and Hongsheng Li and Xiaogang Wang and Yu Qiao},\n  year = {2022},\n  eprint = {2211.05778},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2211.05778}\n}\n","citationPath":"/content/papers/citations/classic-cnns/internimage.4baca0899bc9.bib"},"thumbnail":{"src":"/content/papers/assets/eb597a4ac9a7c2e5e9b05e8ff1c61a951adab71d5cc5a273796762d353a9a0e9.png","width":1012,"height":1179},"locale":"en","path":"/content/papers/en/classic-cnns/internimage.02b85ef97de6.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"where-convolutions-fall-short","text":"Where Convolutions Fall Short","depth":3},{"id":"why-dcnv2-cant-be-used-as-is","text":"Why DCNv2 Can't Be Used As-Is","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"dcnv3s-three-changes","text":"DCNv3's Three Changes","depth":3},{"id":"stacking-it-into-a-network","text":"Stacking It into a Network","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"ablation-the-softmax-change","text":"Ablation: The Softmax Change","depth":3},{"id":"the-cost","text":"The Cost","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/deformable-convnets","relation":"referenced","topic":""},{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/slak","relation":"referenced","topic":""}]},{"id":"classic-cnns/dcnv4","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"DCNv4","subtitle":"Unbounded Weights","author":"Z. Yuan","paperDate":"2024-01-11","publishedOn":"2025-12-29","paperTitle":"Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications","paperUrl":"https://arxiv.org/abs/2401.06197","bibliography":{"title":"Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications","authors":["Yuwen Xiong","Zhiqi Li","Yuntao Chen","Feng Wang","Xizhou Zhu","Jiapeng Luo","Wenhai Wang","Tong Lu","Hongsheng Li","Yu Qiao","Lewei Lu","Jie Zhou","Jifeng Dai"],"url":"https://arxiv.org/abs/2401.06197","pdfUrl":"https://arxiv.org/pdf/2401.06197v1","codeUrl":"https://github.com/OpenGVLab/DCNv4","preprint":{"id":"2401.06197","submittedOn":"2024-01-11","url":"https://arxiv.org/abs/2401.06197"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2024/html/Xiong_Efficient_Deformable_ConvNets_Rethinking_Dynamic_and_Sparse_Operator_for_Vision_CVPR_2024_paper.html"},"citation":"@misc{classiccnnsdcnv42024,\n  title = {{Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications}},\n  author = {Yuwen Xiong and Zhiqi Li and Yuntao Chen and Feng Wang and Xizhou Zhu and Jiapeng Luo and Wenhai Wang and Tong Lu and Hongsheng Li and Yu Qiao and Lewei Lu and Jie Zhou and Jifeng Dai},\n  year = {2024},\n  eprint = {2401.06197},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2401.06197}\n}\n","citationPath":"/content/papers/citations/classic-cnns/dcnv4.f17f87f8ee11.bib"},"thumbnail":{"src":"/content/papers/assets/fe768f9a1b9df098fd7de1ed2c34ba0713cb83936e024d45ca26e85a2b340ff1.png","width":1020,"height":522},"locale":"en","path":"/content/papers/en/classic-cnns/dcnv4.c33fec87d9a7.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"why-is-sampling-just-9-points-still-slow","text":"Why Is Sampling Just 9 Points Still Slow?","depth":3},{"id":"softmax-was-borrowed","text":"Softmax Was Borrowed","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"1-remove-softmax","text":"1. Remove Softmax","depth":3},{"id":"2-move-less-data-on-the-gpu","text":"2. Move Less Data on the GPU","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"the-operator-itself","text":"The Operator Itself","depth":3},{"id":"flashinternimage","text":"FlashInternImage","depth":3},{"id":"swapping-it-into-other-networks","text":"Swapping It into Other Networks","depth":3},{"id":"the-cost","text":"The Cost","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/internimage","relation":"referenced","topic":""},{"id":"classic-cnns/deformable-convnets","relation":"referenced","topic":""}]}],"text":"When we look at a photo, we rarely scan it cell by cell from the upper-left corner. We glance at it, recognize a capybara in a hot spring, then look back at the orange on its head. Cognitive science calls this top-down attention: first get a rough overview, then use it to guide a closer look.\n\nConvolutional backbones work differently. A pyramid architecture downsamples layer by layer, and each layer can use only preceding results. Intermediate layers process features before they know what the whole image is about.\n\nThis paper comes from the University of Hong Kong, by Meng Lou and Yizhou Yu. They want a pure convolutional backbone that also sees the big picture before the details.\n\n## Defining the Problem\n\n### Intermediate Layers Need Direction\n\nUsing Grad-CAM on intermediate layers of Swin-T, ConvNeXt-T, and VMamba-T, the authors find that stage 3, farther from the classifier, often misses objects of the correct class. All three networks can capture long-range relationships, but their intermediate layers lack guidance from high-level semantics.\n\nEarlier vision models have added top-down signals, usually recurrently: feed the final output back to an earlier layer and run it again. That requires another round of computation, an expensive choice for a backbone.\n\n### Kernels Cannot See Far Enough\n\nThe second issue is the operator. Once global semantics are available, the convolution kernel still needs a way to use them.\n\nBoth large kernels and ordinary dynamic convolutions see a region limited by kernel size. As input resolution grows, that region covers a smaller fraction of the image. Deformable convolution reaches farther, but spreading out sampling points loses convolution's local inductive bias. The authors want input-dependent weights that use global information while preserving local bias.\n\n## Solving the Problem\n\n### Split into Three Subnetworks\n\n[![Figure 3 from the paper: Base-Net at the bottom takes a photo of a capybara with an orange on its head through three Embedding and Basic Block groups, reducing resolution from H/4 to H/16. Overview-Net at the upper left follows Base-Net with one Embedding and several Basic Blocks, reducing directly to H/32 and producing a green cube labeled Context Prior. Focus-Net at the upper right also receives Base-Net's output and passes through Dynamic Block, Embedding, and Dynamic Block, from H/16 to H/32. Red context guidance flow arrows deliver the context prior to each Focus-Net Dynamic Block.](/content/papers/assets/d32df0756ccbd0c769259f897b5ad9cf4c8079758897be9a2147cd65a4d46ce1.png)](/content/papers/assets/d32df0756ccbd0c769259f897b5ad9cf4c8079758897be9a2147cd65a4d46ce1.png)\n\n*Figure 3 from Lou and Yu, taken from [**PDF page 4**](https://arxiv.org/pdf/2502.20087v3#page=4), CC BY 4.0. The original caption is omitted.*\n\nThe authors split the network into three subnetworks, calling this the Deep-stage Decomposition Strategy (DDS):\n\n- **Base-Net**: the usual first three stages, downsampling the image to $H/16\\times W/16$ and producing intermediate features.\n- **Overview-Net**: a lightweight path that takes Base-Net's output straight down to $H/32$, quickly forming a rough but semantic understanding of the whole image, called the context prior. This is the overview.\n- **Focus-Net**: a deeper path starting from the same Base-Net output, guided by the context prior at every block. This is the closer look.\n\nThere are effectively two backbones: Base-Net plus Overview-Net, and Base-Net plus Focus-Net, sharing Base-Net. During pretraining, each path has its own classification head and loss. For downstream tasks, Overview-Net no longer needs extra supervision; the feature pyramid combines Base-Net's $H/4$ and $H/8$ features with Focus-Net's $H/16$ and $H/32$.\n\nThe context prior keeps updating through Focus-Net. Each block concatenates it with the feature map, processes the combination, then splits it again and sends the updated prior onward. To keep the signal from fading, the original prior is added back each time: $P_{i+1}=\\alpha P'_i+\\beta P_0$, with learnable scalars $\\alpha$ and $\\beta$.\n\nBase-Net and Overview-Net use a Dilated RepConv from UniRepLKNet: a large kernel reparameterized with small dilated convolutions.\n\n### ContMix: Put the Whole Image into the Kernel\n\nFocus-Net centers on a new dynamic convolution, ContMix (Context-Mixing Dynamic Convolution).\n\n[![Figure 5 from the paper: (a) the ContMix pipeline. Input X passes through a 1×1 convolution and reshaping to form Q. Another path uses adaptive average pooling and a 1×1 convolution to form K. Transposed Q multiplies K, producing a G×HW×S² affinity matrix A, with one row outlined in yellow dashes. That row passes through linear layer W_d and softmax to become a G×1×K² vector, reshaped into a K×K globally aware kernel, then multiplied elementwise with input and summed. A parallel K×K RepConv branch is added to the output. (b) a capybara in a hot spring divided into four regions, each with a colored star at its center. A central 3×3 grid has nine blue globally aware weights, with dashed red lines from the four region centers to the grid center.](/content/papers/assets/99458c439c8f5458671d09ec6d68f52f31a20702b6a0bdba99e2fe33dbf86ea7.png)](/content/papers/assets/99458c439c8f5458671d09ec6d68f52f31a20702b6a0bdba99e2fe33dbf86ea7.png)\n\n*Figure 5 from Lou and Yu, taken from [**PDF page 5**](https://arxiv.org/pdf/2502.20087v3#page=5), CC BY 4.0. The original caption is omitted.*\n\nWalk through panel (a):\n\n1. A 1×1 convolution maps input $X$ to $Q$, giving one query vector at each of $HW$ locations.\n2. A separate path adaptively average-pools $X$ into $S\\times S$ region centers, then applies a 1×1 convolution to obtain $K$. The paper uses $S=7$, giving 49 centers.\n3. Multiplying $Q$ and $K$ gives an affinity matrix $A$ of size $HW\\times S^2$. Row $i$ records how similar location $i$ is to each of the 49 region centers.\n4. Linear layer $W_d$ maps each row from 49 numbers to $K^2$, followed by softmax. This becomes that location's own $K\\times K$ kernel.\n5. Convolve the input with these kernels, using a dedicated one at every location.\n\nChannels are divided into $G$ groups, each computing its own affinity and kernel, shared within that group. In OverLoCK's dynamic blocks, $Q$ comes from the feature map and $K$ from the context prior. This is where Overview-Net's global understanding enters the kernels.\n\nStep 4 is the key. Every kernel weight is computed from the relationship between one location and 49 centers across the image, so it contains global information. Yet the convolution still operates within a local $K\\times K$ neighborhood, preserving local inductive bias. Panel (b) illustrates this: the 3×3 grid stays in place, while region centers in the four corners determine its weights.\n\nWrite a minimal version to check this. Change a pixel at the upper left and see whether output at the lower right changes:\n\n```python\nimport torch\nimport torch.nn.functional as F\n\ntorch.manual_seed(0)\nC, H, W = 8, 16, 16\nS, K, G = 4, 3, 2                    # 4×4 region centers, 3×3 dynamic kernels, 2 channel groups\nWq = torch.nn.Conv2d(C, C, 1)\nWk = torch.nn.Conv2d(C, C, 1)\nWd = torch.nn.Linear(S * S, K * K)   # Map 16 similarities to 9 kernel weights\n\ndef contmix(x):\n    q = Wq(x[None])[0].flatten(1)                                   # C × HW: queries at each location\n    k = Wk(F.adaptive_avg_pool2d(x[None], S))[0].flatten(1)         # C × S²: region centers\n    out = []\n    for qg, kg, xg in zip(q.chunk(G), k.chunk(G), x.chunk(G)):\n        A = qg.T @ kg                                               # HW × S²: similarity of each location to each center\n        D = Wd(A).softmax(-1)                                       # HW × K²: a dedicated 3×3 kernel at each location\n        patches = F.unfold(xg[None], K, padding=K // 2)[0]          # 3×3 neighbors around each location\n        patches = patches.view(C // G, K * K, H * W)\n        out.append((patches * D.T[None]).sum(1).view(C // G, H, W))\n    return torch.cat(out)\n\nstatic = torch.nn.Conv2d(C, C, K, padding=K // 2, groups=C)       # Ordinary 3×3 depthwise convolution\n\nx = torch.randn(C, H, W)\nx2 = x.clone()\nx2[:, 0, 0] += 5.0                                                  # Change only the upper-left pixel\n\nwith torch.no_grad():\n    for name, f in [(\"3×3 depthwise convolution\", lambda t: static(t[None])[0]), (\"ContMix (3×3)\", contmix)]:\n        diff = (f(x2) - f(x))[:, 12, 12].abs().max().item()       # Measure the output change at lower-right position (12, 12)\n        print(f\"{name}: change in lower-right output {diff:.4f}\")\n```\n\n```text\n3×3 depthwise convolution: change in lower-right output 0.0000\nContMix (3×3): change in lower-right output 0.0274\n```\n\nBoth use 3×3 kernels, yet ordinary convolution cannot see the change 12 positions away; ContMix can. The upper-left pixel changes the first region center, changing every location's affinities and therefore the lower-right location's kernel.\n\nThe actual module is more complex. The official code includes a value projection and relative position bias. Half the groups use large kernels; the other half use small 5×5 kernels, with a Dilated RepConv branch in parallel. If the neighborhood-attention package NATTEN is installed, the dynamic-kernel application directly uses its CUDA operator: the weighted sum is identical to neighborhood attention's. The difference is where the weights come from. Neighborhood attention multiplies queries by neighboring keys; ContMix multiplies queries by region centers across the image.\n\n### Four Sizes\n\nThe four variants are XT, T, S, and B. T has 33M parameters and 5.5 GFLOPs; B has 95M and 16.7 GFLOPs. Base-Net's three stages use kernels 17, 15, and 13; Overview-Net uses 7; Focus-Net uses 13 and 7.\n\n## Discussion\n\n### ImageNet\n\nTraining on ImageNet-1K follows DeiT for 300 epochs. Throughput is measured on one L40S, at batch size 128 with 224×224 inputs:\n\n| Model | Parameters | FLOPs | Throughput (images/s) | top-1 |\n| --- | ---: | ---: | ---: | ---: |\n| ConvNeXt-T | 29M | 4.5G | 1507 | 82.1% |\n| UniRepLKNet-T | 31M | 4.9G | 1094 | 83.2% |\n| InternImage-T | 30M | 5.0G | 1084 | 83.5% |\n| MogaNet-S | 25M | 5.0G | 766 | 83.4% |\n| OverLoCK-T | 33M | 5.5G | 810 | **84.2%** |\n| ConvNeXt-S | 50M | 8.7G | 926 | 83.1% |\n| ConvNeXt-B | 89M | 15.4G | 608 | 83.8% |\n| OverLoCK-S | 56M | 9.7G | 480 | **84.8%** |\n| OverLoCK-B | 95M | 16.7G | 306 | **85.1%** |\n\n*Table: selected rows from the paper's Table 2 and appendix Table I.*\n\nThe abstract emphasizes that OverLoCK-T beats ConvNeXt-B with one third the parameters and FLOPs, and higher throughput, 810 versus 608. That is true.\n\nRemember the T label, though. ConvNeXt-T handles 1507 images per second, while OverLoCK-T manages a little over half that. Similar-speed comparisons are fairer: ConvNeXt-S runs at 926 images/s and 83.1%; OverLoCK-T is a little over 10% slower and gains 1.1 percentage points.\n\n### Detection and Segmentation\n\nDownstream gaps are larger than classification gaps. On COCO with Mask R-CNN trained for 12 epochs, OverLoCK-T gets 48.3 box AP, against 47.2 for InternImage-T and 44.2 for ConvNeXt-T. On ADE20K with UperNet, OverLoCK-T reaches 50.3 single-scale mIoU, against 48.6 for UniRepLKNet-T and 46.0 for ConvNeXt-T.\n\nDetection and segmentation use much higher input resolutions than classification. ContMix keeps a fixed 7×7 set of region centers covering the full image at any size. This is where it pulls ahead of fixed-size large kernels.\n\n### Add One Thing at a Time\n\nThe authors start from a purely static convolutional network and add one component at a time. These experiments train only 120 epochs on ImageNet and 80K steps on ADE20K, so the values cannot be directly compared with the main tables:\n\n| Model | FLOPs | top-1 | mIoU |\n| --- | ---: | ---: | ---: |\n| Static baseline | 2.6G | 78.5% | 41.1 |\n| Add recurrent feedback | 4.4G | 76.8% | 39.5 |\n| Switch to three DDS subnetworks | 2.6G | 79.0% | 41.6 |\n| Use ContMix dynamic blocks in Focus-Net | 2.6G | 80.0% | 42.9 |\n| Add Overview-Net auxiliary loss | 2.6G | 80.2% | 43.1 |\n| Reintroduce the original context prior at every block | 2.6G | 80.4% | 43.4 |\n| Add gating (OverLoCK-XT) | 2.6G | **80.8%** | **43.8** |\n\n*Table: selected rows from the paper's Table 6.*\n\nA few findings are clear.\n\n1. **Recurrent feedback fails here**: following AbsViT and feeding stage 4 back into stage 3 adds 70% compute and lowers accuracy.\n2. **DDS helps at matched compute**: top-1 and mIoU each gain 0.5, adding top-down guidance without another pass.\n3. **ContMix is the largest step**: replacing static blocks with dynamic ones adds 1.0 top-1 and 1.3 mIoU.\n\nThe appendix adds another comparison. Remove Overview-Net and use a standard four-stage pyramid, keeping the blocks unchanged: top-1 falls from 80.7% to 79.2% and mIoU from 43.8 to 41.9. That is the contribution of the top-down structure itself.\n\n### Against Other Dynamic Operators\n\nTo compare operators fairly, the authors build a separate Swin-shaped network and change only the token mixer, using 7×7 kernels or windows throughout:\n\n| Operator | top-1 | ADE20K mIoU |\n| --- | ---: | ---: |\n| DyConv | 77.5% | 36.3 |\n| Involution | 78.3% | 37.5 |\n| Shifted-window attention (Swin) | 78.4% | 37.4 |\n| Neighborhood attention (NATTEN) | 78.6% | 38.1 |\n| DCNv3 | 78.7% | 37.5 |\n| ContMix | **78.8%** | **39.0** |\n\n*Table: selected rows from the paper's Table 7.*\n\nClassification is close, with ContMix just 0.1 ahead of second place. At higher segmentation resolution, its lead ranges from 0.9 to 2.7.\n\nAn aside on softmax: ContMix uses it when generating kernels. Removing it in an appendix ablation changes top-1 from 80.8% to 80.5% and mIoU from 43.8 to 43.5, a small effect. In InternImage, replacing softmax with sigmoid destabilizes training; DCNv4 converges faster after removing softmax. Softmax carries very different weight across these dynamic operators.\n\n\n> [!TIP]\n> \n> For DCNv3 and DCNv4's different uses of softmax, see:\n> \n> - **[[22.11] InternImage: Miracles by Brute Force?](/blog/papers/classic-cnns/internimage)**\n> - **[[24.01] DCNv4: Unbounded Weights](/blog/papers/classic-cnns/dcnv4)**\n\n\n### Using It in Practice\n\nThe official code has several external dependencies. NATTEN is recommended for dynamic convolution; it runs without it, more slowly. Large kernels can use UniRepLKNet's iGEMM implementation. Detection and segmentation also need mmcv. Many GitHub issues concern installation.\n\nThe accelerated large-kernel implementation also depends on batch size. A user [reports](https://github.com/LMMMEng/OverLoCK/issues/33) training becoming three times slower after enabling `use_gemm`. The author replies that the implementation has no advantage at small batches; disable it there, with no effect on accuracy. Another user [training COCO on 4 A100s at total batch size 8](https://github.com/LMMMEng/OverLoCK/issues/38) estimates 41 days. The authors take two or three days on 8 H800s and recommend larger batches and gradient checkpointing.\n\n## Conclusion\n\nOverLoCK's two ideas can be considered separately. DDS uses a lightweight path to see the whole image, then sends that understanding to every later block. ContMix determines each location's kernel from its relationship to global region centers, while keeping convolution local. The former gains 0.5 at matched compute; the latter gains 1.0 or more, with larger gaps in detection and segmentation.\n\n“See the big picture first” sounds like the headline idea, yet the ablations show that the largest contribution comes from ContMix putting global information into kernel weights. The top-down structure supplies that information; ContMix gives it somewhere to go.\n\nTo use it, prepare the environment first: NATTEN, the large-kernel CUDA extension, and mmcv. At small batch sizes, remember to turn `use_gemm` off, or it will only slow things down.\n","imageSizes":{"/content/papers/assets/d32df0756ccbd0c769259f897b5ad9cf4c8079758897be9a2147cd65a4d46ce1.png":{"width":1047,"height":762},"/content/papers/assets/99458c439c8f5458671d09ec6d68f52f31a20702b6a0bdba99e2fe33dbf86ea7.png":{"width":2112,"height":681}}}