{"notes":[{"id":"classic-cnns/wbmm","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"WBMM","subtitle":"窗开越大越快","author":"Z. Yuan","paperDate":"2026-07-02","publishedOn":"2026-10-09","paperTitle":"WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution","paperUrl":"https://arxiv.org/abs/2607.02097","bibliography":{"title":"WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution","authors":["Wan Song","Wei Zhou","Rui Wang","Jun Yu","Toru Kurihara","Jiajia Xu","Shu Zhan"],"url":"https://arxiv.org/abs/2607.02097","pdfUrl":"https://arxiv.org/pdf/2607.02097v2","codeUrl":"https://github.com/wansong-s/WBMM","preprint":{"id":"2607.02097","submittedOn":"2026-07-02","url":"https://arxiv.org/abs/2607.02097"},"publication":{"venue":"ICML","year":2026,"kind":"conference","url":"https://proceedings.mlr.press/v306/song26ae.html"},"citation":"@misc{classiccnnswbmm2026,\n  title = {{WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution}},\n  author = {Wan Song and Wei Zhou and Rui Wang and Jun Yu and Toru Kurihara and Jiajia Xu and Shu Zhan},\n  year = {2026},\n  eprint = {2607.02097},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2607.02097}\n}\n","citationPath":"/content/papers/citations/classic-cnns/wbmm.5eb4a1ff4a34.bib"},"thumbnail":{"src":"/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png","width":1017,"height":360},"locale":"zh-hans","path":"/content/papers/zh-hans/classic-cnns/wbmm.a876cf8af1ec.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"卷积在等数据","text":"卷积在等数据","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"把卷积写成矩阵乘法","text":"把卷积写成矩阵乘法","depth":3},{"id":"窗口越大越快","text":"窗口越大越快","depth":3},{"id":"补上窗口之间的交流","text":"补上窗口之间的交流","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"检测与分割","text":"检测与分割","depth":3},{"id":"其他硬件","text":"其他硬件","depth":3},{"id":"限制","text":"限制","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/slak","relation":"referenced","topic":""},{"id":"reparameterization/unireplknet","relation":"referenced","topic":""}]},{"id":"reparameterization/replknet","category":"reparameterization","legacyPath":"reparameterization/2203-replknet","legacySlug":"2203-replknet","legacyPaths":[],"title":"RepLKNet","subtitle":"巨无霸卷积核","author":"Z. Yuan","paperDate":"2022-03-13","publishedOn":"2024-07-21","paperTitle":"Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs","paperUrl":"https://arxiv.org/abs/2203.06717","bibliography":{"title":"Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs","authors":["Xiaohan Ding","Xiangyu Zhang","Yizhuang Zhou","Jungong Han","Guiguang Ding","Jian Sun"],"url":"https://arxiv.org/abs/2203.06717","pdfUrl":"https://arxiv.org/pdf/2203.06717","codeUrl":null,"preprint":{"id":"2203.06717","submittedOn":"2022-03-13","url":"https://arxiv.org/abs/2203.06717"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://arxiv.org/abs/2203.06717"},"citation":"@misc{reparameterizationreplknet2022,\n  title = {{Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs}},\n  author = {Xiaohan Ding and Xiangyu Zhang and Yizhuang Zhou and Jungong Han and Guiguang Ding and Jian Sun},\n  year = {2022},\n  eprint = {2203.06717},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2203.06717}\n}\n","citationPath":"/content/papers/citations/reparameterization/replknet.c7cc2407aa19.bib"},"thumbnail":{"src":"/content/papers/assets/b85fa678b6e815b9d112b67d68443295a98c80478664ec8053c2e832d89574ca.jpg","width":1140,"height":972},"locale":"zh-hans","path":"/content/papers/zh-hans/reparameterization/replknet.63f1e846afbb.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"大核卷积使用说明书","text":"大核卷积使用说明书","depth":2},{"id":"深度大核卷积仍然高效","text":"深度大核卷积仍然高效","depth":3},{"id":"大核卷积十分依赖残差连接","text":"大核卷积十分依赖残差连接","depth":3},{"id":"大核卷积需要重参数化","text":"大核卷积需要重参数化","depth":3},{"id":"大核卷积明显提升下游任务性能","text":"大核卷积明显提升下游任务性能","depth":3},{"id":"大核卷积对于小特征图的效果依然很好","text":"大核卷积对于小特征图的效果依然很好","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"网络架构","text":"网络架构","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"大核卷积评估","text":"大核卷积评估","depth":3},{"id":"imagenet-上的性能","text":"ImageNet 上的性能","depth":3},{"id":"大核-cnn-比深度小核模型有更大的-erf","text":"大核 CNN 比深度小核模型有更大的 ERF","depth":3},{"id":"形状偏差","text":"形状偏差","depth":3},{"id":"限制","text":"限制","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"comparison","topic":"大卷积核与简化骨架"}]},{"id":"classic-cnns/slak","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"SLaK","subtitle":"卷积核 Pro Max","author":"Z. Yuan","paperDate":"2022-07-07","publishedOn":"2025-03-09","world":"external/slak","paperTitle":"More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity","paperUrl":"https://arxiv.org/abs/2207.03620","bibliography":{"title":"More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity","authors":["Shiwei Liu","Tianlong Chen","Xiaohan Chen","Xuxi Chen","Qiao Xiao","Boqian Wu","Tommi Kärkkäinen","Mykola Pechenizkiy","Decebal Mocanu","Zhangyang Wang"],"url":"https://arxiv.org/abs/2207.03620","pdfUrl":"https://arxiv.org/pdf/2207.03620v3","codeUrl":null,"preprint":{"id":"2207.03620","submittedOn":"2022-07-07","url":"https://arxiv.org/abs/2207.03620"},"publication":{"venue":"ICLR","year":2023,"kind":"conference","url":"https://iclr.cc/virtual/2023/poster/11529"},"citation":"@misc{classiccnnsslak2022,\n  title = {{More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity}},\n  author = {Shiwei Liu and Tianlong Chen and Xiaohan Chen and Xuxi Chen and Qiao Xiao and Boqian Wu and Tommi Kärkkäinen and Mykola Pechenizkiy and Decebal Mocanu and Zhangyang Wang},\n  year = {2022},\n  eprint = {2207.03620},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2207.03620}\n}\n","citationPath":"/content/papers/citations/classic-cnns/slak.e167e693df01.bib"},"thumbnail":{"src":"/content/papers/assets/b5b8ba279efe42831c7ec4e60395d0559da26774c8519a438783f49f4b814a85.png","width":1640,"height":660},"locale":"zh-hans","path":"/content/papers/zh-hans/classic-cnns/slak.d84cfaf376c9.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"31-之后","text":"31 之后","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"第一步拆成两根长条","text":"第一步：拆成两根长条","depth":3},{"id":"第二步稀疏一点宽一点","text":"第二步：稀疏一点，宽一点","depth":3},{"id":"slak-的样子","text":"SLaK 的样子","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"下游任务","text":"下游任务","depth":3},{"id":"实际跑起来","text":"实际跑起来","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/resnext","relation":"referenced","topic":""}]},{"id":"reparameterization/unireplknet","category":"reparameterization","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"UniRepLKNet","subtitle":"大核跨界","author":"Z. Yuan","paperDate":"2023-11-27","publishedOn":"2025-12-14","world":"external/unireplknet","paperTitle":"UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition","paperUrl":"https://arxiv.org/abs/2311.15599","bibliography":{"title":"UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition","authors":["Xiaohan Ding","Yiyuan Zhang","Yixiao Ge","Sijie Zhao","Lin Song","Xiangyu Yue","Ying Shan"],"url":"https://arxiv.org/abs/2311.15599","pdfUrl":"https://arxiv.org/pdf/2311.15599v2","codeUrl":"https://github.com/AILab-CVC/UniRepLKNet","preprint":{"id":"2311.15599","submittedOn":"2023-11-27","url":"https://arxiv.org/abs/2311.15599"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2024/html/Ding_UniRepLKNet_A_Universal_Perception_Large-Kernel_ConvNet_for_Audio_Video_Point_CVPR_2024_paper.html"},"citation":"@misc{reparameterizationunireplknet2023,\n  title = {{UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition}},\n  author = {Xiaohan Ding and Yiyuan Zhang and Yixiao Ge and Sijie Zhao and Lin Song and Xiangyu Yue and Ying Shan},\n  year = {2023},\n  eprint = {2311.15599},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2311.15599}\n}\n","citationPath":"/content/papers/citations/reparameterization/unireplknet.df5ca5b9a0be.bib"},"thumbnail":{"src":"/content/papers/assets/8dc0f2f7d7d64f92bea8485e2e00428225297bb13404dee50fe50f28385d34c8.png","width":1680,"height":384},"locale":"zh-hans","path":"/content/papers/zh-hans/reparameterization/unireplknet.6c5d5c7f5107.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"一层-33-做了三件事","text":"一层 3×3 做了三件事","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"基准架构","text":"基准架构","depth":3},{"id":"准则一加深用便宜的结构","text":"准则一：加深用便宜的结构","depth":3},{"id":"准则二用空洞小卷积重参数化","text":"准则二：用空洞小卷积重参数化","depth":3},{"id":"准则三卷积核大小看下游任务","text":"准则三：卷积核大小看下游任务","depth":3},{"id":"准则四加深时加的是小卷积核","text":"准则四：加深时，加的是小卷积核","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"图像","text":"图像","depth":3},{"id":"其他模态","text":"其他模态","depth":3},{"id":"代价","text":"代价","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/slak","relation":"referenced","topic":""},{"id":"lightweight/senet","relation":"referenced","topic":""}]}],"text":"卷积核越做越大。ConvNeXt 用 7×7，RepLKNet 做到 31×31，SLaK 做到 51×51，UniRepLKNet 回头选了 13×13。准确率一路往上，速度问题却一直没解决。\n\ndepthwise 卷积的 FLOPs 很少，卷积核放大之后却慢得很快。这篇的作者量过，标准的 depthwise 卷积从 5×5 放大到 13×13，吞吐量掉了 71% 到 78%；放大到 27×27，掉了 92% 到 94%。不管批量大小、特征图多大，结果都差不多。\n\nRepLKNet 和 UniRepLKNet 为此写了专用的 CUDA 算子，论文里把它叫做 LKA（Large Kernel Acceleration）。它在小特征图上有用，可是特征图一大反而更慢。在 224×224、批量 128 的设置下，LKA 的 7×7 比不加速的 5×5 慢了 80%，13×13 慢了 89%。检测、分割、超分辨率这些任务，偏偏都要处理大特征图。\n\n这篇来自合肥工业大学、Lingyang Industrial Internet 公司和日本高知工科大学。他们换了一个方向，不去优化卷积怎么读数据，而是让卷积改用矩阵乘法来算。\n\n\n> [!TIP]\n> \n> 大卷积核这条路上的几站，可以参考我们的笔记：\n> \n> - **[[22.03] RepLKNet：巨无霸卷积核](/blog/papers/reparameterization/replknet)**\n> - **[[22.07] SLaK：卷积核 Pro Max](/blog/papers/classic-cnns/slak)**\n> - **[[23.11] UniRepLKNet：大核跨界](/blog/papers/reparameterization/unireplknet)**\n\n\n## 定义问题\n\n### 卷积在等数据\n\n[![原论文 Figure 2：四个面板。(a) 5×5 depthwise 卷积，一个红色的输出位置 y 从周围 5 行、每行 5 个格子收集数值，红色细线从四面八方连到 y，注明每个输出要从 5 个不连续的行收集 25 个值。(b) WBMM 的 4×4 窗口，8×8 的特征图被切成 W0 到 W3 四个不重叠的彩色区块，注明每个窗口从一个连续的区块读 16 个值。(c) depthwise 卷积的内存访问，第 1 到第 5 行各有一段红框，标示 fetch #1 到 fetch #5，下方列出 5 次独立读取、跨距等于宽度、工作集横跨多条缓存线、每个字节 O(1) 次运算，属于内存受限。(d) WBMM 的内存访问，四行中各取 4 个值的蓝框，经过 window_partition 和 reshape 之后排成一条连续的 16 格，只要 fetch #1 一次，下方列出一次连续读取、地址连续、窗口放得进 L1 缓存、每个字节 O(w²) 次运算，属于计算受限。](/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png)](/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png)\n\n*原论文 Figure 2（Song 等人），取自 [**PDF 第 2 页**](https://arxiv.org/pdf/2607.02097v2#page=2)。原图注未收录。*\n\n问题出在内存访问。特征图在内存里是一行接一行排的，depthwise 卷积的每个输出，要从上下 $k$ 行各抓 $k$ 个值。这些值在内存里不连续，同一行的 $k$ 个值相邻，换到下一行就要跳过整整一行的宽度。\n\n卷积核越大，要跳的行越多，缓存越装不下。每读进一个值只做一次乘加，GPU 的算力用不上，大部分时间在等数据。\n\n## 解决问题\n\n### 把卷积写成矩阵乘法\n\n[![原论文 Figure 1：左半 (a) 是 depthwise 卷积，输入 X 是一张大网格，中间一个红色格子 (i, j) 周围框出 7×7 的橘色范围，和 7×7 的卷积核 W 做卷积，箭头标示收集 7² = 49 个邻居，得到输出 y[i, j]。右半 (b) 是 WBMM，输入 X 被切成四个颜色的 7×7 窗口；上方一条浅蓝色的相对位置表 R，标示大小为 C × (2·7 − 1)² = C × 169，经过 index 箭头组成一个 C × 49 × 49 的矩阵 M；输入窗口经过标示 Batched MatMul 的绿色箭头和 M 相乘，得到同样切成四个窗口的输出 Y。](/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png)](/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png)\n\n*原论文 Figure 1（Song 等人），取自 [**PDF 第 1 页**](https://arxiv.org/pdf/2607.02097v2#page=1)。原图注未收录。*\n\nWBMM（Windowed Batch Matrix Multiplication）分三步：\n\n1. **切窗口**：把特征图切成不重叠的 $w\\times w$ 窗口，例如 7×7，每个窗口摊平成 49 个值，在内存里排成连续的一段。\n2. **查表组权重**：每个通道有一张相对位置表 $R$，大小是 $(2w-1)^2$，7×7 窗口就是 13×13 = 169 个参数。窗口里第 $i$ 个位置对第 $j$ 个位置的权重，只看两者的相对位置，去表里查出来，组成一个 49×49 的矩阵 $M$。\n3. **批量矩阵乘法**：同一个通道的所有窗口共用同一个 $M$，所以把通道当成批量维度，一次 `bmm` 就算完所有窗口。\n\n权重只看相对位置，这正是卷积的定义。论文的定理 3.2 证明，在有界的特征图上，depthwise 卷积可以精确写成依相对位置决定权重的矩阵乘法。套到每个窗口上，WBMM 就等于一个 13×13 的 depthwise 卷积，窗口外一律当成 0。我们照着写一遍，和逐窗口做 13×13 卷积的结果比一比：\n\n```python\nimport torch\nimport torch.nn.functional as F\n\ntorch.manual_seed(0)\nC, H, W, w = 4, 14, 14, 7                       # 4 个通道、14×14 的特征图、7×7 的窗口\nd = w * w                                        # 每个窗口 49 个位置\nR = torch.randn(C, (2 * w - 1) ** 2)             # 相对位置表：每个通道 13×13 = 169 个参数\n\n# 相对位置索引：窗口里第 i 个位置看第 j 个位置，该用表里的哪一格\npos = torch.stack(torch.meshgrid(torch.arange(w), torch.arange(w), indexing=\"ij\")).flatten(1)  # 2 × 49\nrel = pos[:, None, :] - pos[:, :, None] + (w - 1)                                                # 2 × 49 × 49\nindex = rel[0] * (2 * w - 1) + rel[1]\n\nM = R[:, index]                                  # C × 49 × 49：查表组出权重矩阵\n\n# 把特征图切成不重叠的窗口，排成 (C, 窗口数, 49)\nx = torch.randn(1, C, H, W)\nnh, nw = H // w, W // w\nwindows = x.view(C, nh, w, nw, w).permute(0, 1, 3, 2, 4).reshape(C, nh * nw, d)\ny = torch.bmm(windows, M.transpose(1, 2))         # 每个通道一次批量矩阵乘法，算完所有窗口\ny = y.view(C, nh, nw, w, w).permute(0, 1, 3, 2, 4).reshape(1, C, H, W)\n\n# 对照：13×13 的 depthwise 卷积，一个窗口一个窗口算，窗口外补 0\nkernel = R.view(C, 1, 2 * w - 1, 2 * w - 1)\nref = torch.zeros_like(x)\nfor i in range(nh):\n    for j in range(nw):\n        patch = x[..., i * w:(i + 1) * w, j * w:(j + 1) * w]\n        ref[..., i * w:(i + 1) * w, j * w:(j + 1) * w] = F.conv2d(patch, kernel, padding=w - 1, groups=C)\n\nprint(\"权重矩阵：\", tuple(M.shape), \"，由\", R.shape[1], \"个参数查表组成\")\nprint(\"和窗口内 13×13 depthwise 卷积的最大差距：\", f\"{(y - ref).abs().max():.1e}\")\n```\n\n```text\n权重矩阵： (4, 49, 49) ，由 169 个参数查表组成\n和窗口内 13×13 depthwise 卷积的最大差距： 2.9e-06\n```\n\n两种算法的结果一样，差距只是浮点数误差。差别在内存里发生的事，卷积要一个一个去收集散落的邻居，WBMM 读一整块连续的窗口，交给 GPU 最擅长的矩阵乘法。\n\n### 窗口越大越快\n\n这带来一个反直觉的性质。depthwise 卷积的卷积核越大越慢，WBMM 却是窗口越大越快。\n\n一个 $w\\times w$ 的窗口要读 $w^2$ 个值，矩阵乘法要做 $w^4$ 次乘加，所以每读一个值能做 $w^2$ 次运算。窗口越大，计算和读取的比例越高，GPU 越有事做。权重矩阵 $M$ 虽然有 $w^4$ 个元素，但整批数据的所有窗口共用，读一次就摊掉了。\n\n14×14 的窗口，每个输出要做 196 次乘加，是 5×5 depthwise 卷积的将近 8 倍，看到的范围也是 7.8 倍。论文的算子测试在 A800 上用 FP32、256 个通道，WBMM 的 14×14 窗口和标准的 5×5 depthwise 卷积比：\n\n| 批量 | 14×14 特征图 | 28×28 | 56×56 | 112×112 | 224×224 |\n| ---: | ---: | ---: | ---: | ---: | ---: |\n| 4 | 0.45× | 0.37× | 1.41× | 1.64× | 2.02× |\n| 16 | 0.62× | 1.56× | 1.80× | 2.00× | 2.07× |\n| 64 | 1.86× | 1.85× | 2.01× | 2.01× | 2.03× |\n| 128 | 3.17× | 2.00× | 2.06× | 1.97× | 2.01× |\n| 256 | 3.72× | 2.02× | 2.02× | 1.95× | — |\n\n*表：摘自原论文 Table 1，数字是 WBMM 相对 5×5 depthwise 卷积的速度倍数。*\n\n特征图够大、批量够大时，WBMM 的速度大约是基准的 2 倍。批量小、特征图又小的时候反而比较慢，批量 4、28×28 只有 0.37 倍。\n\n这和窗口注意力很不一样。Swin 的窗口注意力每个样本、每个窗口都要算一个依输入变化的矩阵，再做 softmax；WBMM 的 $M$ 和输入无关，整批共用一个，推理时还可以先算好存起来。\n\n### 补上窗口之间的交流\n\n窗口不重叠，信息出不了窗口。作者在 WBMM 区块之间穿插一般的 3×3 depthwise 卷积，让相邻窗口交换信息。\n\n对分割这类需要多尺度的任务，还有一个分层式的窗口重参数化。训练时同时有一个 14×14 的大窗口和四个 7×7 的小窗口，各自有相对位置表；推理前把小窗口的矩阵加进大窗口矩阵的对角块，合成一个矩阵，推理时没有额外成本。这和 UniRepLKNet 把小卷积核合进大卷积核是同一个思路。\n\n整个模型以 UniRepLKNet 为基准，把大卷积核换成 WBMM，再调整各阶段里 WBMM 区块和 3×3 区块的排法。\n\n## 讨论\n\n### ImageNet\n\n| 模型 | 每个 epoch 的训练时间 | 训练内存 | top-1 |\n| --- | ---: | ---: | ---: |\n| WBMM-T | **6:18** | **15.0 GB** | 83.2% |\n| UniRepLKNet-T（有 LKA） | 9:03 | 16.6 GB | 83.2% |\n| UniRepLKNet-T（没有 LKA） | 11:03 | 16.6 GB | 83.2% |\n| WBMM-S | **8:35** | **20.4 GB** | 83.9% |\n| UniRepLKNet-S（有 LKA） | 12:10 | 22.3 GB | 83.9% |\n\n*表：摘自原论文 Table 5 的部分行。8 张 A800。*\n\n准确率完全一样，训练速度达到基准的 1.4 倍以上，不用 LKA 的话差距更大。四个尺寸算下来，训练加速是 1.31 到 1.88 倍，这是论文摘要里的数字。\n\n不过这张表比的是 UniRepLKNet 整套，它训练时还挂着好几条空洞小卷积的重参数化分支。附录有一张更公平的表，所有东西都一样，只换空间算子：\n\n| 空间算子 | 每个 epoch | top-1 | ADE20K mIoU | 推理（张／秒） |\n| --- | ---: | ---: | ---: | ---: |\n| WBMM（7×7 窗口） | **6:20** | 83.0% | 48.3 | **1833** |\n| WBMM（14×14 分层式） | 6:31 | 83.2% | **48.8** | 1842 |\n| 13×13 depthwise ＋ LKA | 6:43 | 83.1% | 48.3 | 1662 |\n| 13×13 depthwise（没有 LKA） | 9:11 | 83.1% | 48.3 | 1305 |\n| 13×13 depthwise ＋ 重参数化 ＋ LKA | 10:21 | 83.2% | 48.7 | 1662 |\n| SLaK 原版 | 22:03 | 83.3% | 47.4 | 549 |\n| 7×7 窗口注意力 | 8:38 | 83.3% | 48.1 | 1245 |\n\n*表：摘自原论文 Table 8 的部分行，全部是 Tiny 尺寸、同一套架构。*\n\n和一般的 13×13 depthwise 卷积加 LKA 比，WBMM 的训练只快了 6%，推理快了 10%。前一张表那 1.4 倍，大半来自拿掉 UniRepLKNet 的多分支重参数化，10:21 变成 6:20。和没有 LKA 的 depthwise 卷积比，WBMM 的速度才达到基准的 1.4 倍以上。\n\nWBMM 真正的价值在于，不用写任何专用的 CUDA 算子，就达到甚至超过 LKA 的速度。\n\n### 检测与分割\n\n| 方法 | ADE20K mIoU（单尺度） | COCO box AP | COCO mask AP |\n| --- | ---: | ---: | ---: |\n| UniRepLKNet-T | 48.6 | 51.8 | 44.9 |\n| WBMM-T（7×7） | 48.3 | 51.6 | 44.8 |\n| WBMM-T（分层式） | **48.8** | **51.9** | **45.1** |\n\n*表：摘自原论文 Table 6 与 Table 7。ADE20K 用 UperNet，COCO 用 Cascade Mask R-CNN。*\n\n纯 7×7 窗口的版本比 UniRepLKNet 低 0.1 到 0.3；加上分层式重参数化，才比它高 0.1 到 0.2，参数也多了 3M 到 4M。分割和检测上两者大致打平，WBMM 赢在速度。\n\n### 其他硬件\n\n作者也在其他设备上量了速度。A800 上比有 LKA 的 UniRepLKNet 快 1.01 到 1.28 倍；Intel i7-13700K CPU 上没有 LKA 可用，WBMM 靠 BLAS 库快了 1.03 到 1.48 倍；Jetson Orin Nano 的 GPU 用 FP16 快了 1.19 到 3.12 倍，CPU 用 INT8 快了 1.33 到 2.44 倍。矩阵乘法是每种硬件都优化过的运算，这点在边缘设备上特别划算。\n\n### 限制\n\n作者在附录列了几个限制。权重矩阵 $M$ 的大小随窗口边长的四次方增长，14×14 窗口时一个 256 通道的层大约 37.5 MB，超过 28×28 就放不下了。窗口大小要看任务挑，纯 14×14 在 ADE20K 上反而比纯 7×7 差。权重和输入无关，需要依内容调整的任务可能不够用。\n\n## 结论\n\nWBMM 没有改变模型看到什么。在每个窗口里，它就是一个 13×13 的 depthwise 卷积；它改的是数据在内存里怎么排，以及用哪一种算子去算。卷积换成批量矩阵乘法之后，从等数据变成忙计算，窗口越大反而越快。\n\n读表时要分清楚比的是谁。摘要里那个 1.31 到 1.88 倍，是和整套 UniRepLKNet 比；只换算子的话，和加了 LKA 的 depthwise 卷积比，训练快的是 6%。它真正省下的，是那套专用的 CUDA 算子。\n\n所以如果你的大卷积核模型卡在高分辨率的速度，不妨先试这招。只要 PyTorch 的 `bmm`，上面示范程序的前半段就是全部的核心逻辑，连 CUDA 扩展都不用编。\n","imageSizes":{"/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png":{"width":1032,"height":864},"/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png":{"width":1017,"height":360}}}