{"notes":[{"id":"classic-cnns/wbmm","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"WBMM","subtitle":"窗開越大越快","author":"Z. Yuan","paperDate":"2026-07-02","publishedOn":"2026-10-09","paperTitle":"WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution","paperUrl":"https://arxiv.org/abs/2607.02097","bibliography":{"title":"WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution","authors":["Wan Song","Wei Zhou","Rui Wang","Jun Yu","Toru Kurihara","Jiajia Xu","Shu Zhan"],"url":"https://arxiv.org/abs/2607.02097","pdfUrl":"https://arxiv.org/pdf/2607.02097v2","codeUrl":"https://github.com/wansong-s/WBMM","preprint":{"id":"2607.02097","submittedOn":"2026-07-02","url":"https://arxiv.org/abs/2607.02097"},"publication":{"venue":"ICML","year":2026,"kind":"conference","url":"https://proceedings.mlr.press/v306/song26ae.html"},"citation":"@misc{classiccnnswbmm2026,\n  title = {{WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution}},\n  author = {Wan Song and Wei Zhou and Rui Wang and Jun Yu and Toru Kurihara and Jiajia Xu and Shu Zhan},\n  year = {2026},\n  eprint = {2607.02097},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2607.02097}\n}\n","citationPath":"/content/papers/citations/classic-cnns/wbmm.5eb4a1ff4a34.bib"},"thumbnail":{"src":"/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png","width":1017,"height":360},"locale":"zh-hant","path":"/content/papers/zh-hant/classic-cnns/wbmm.02a87a6f9851.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"卷積在等資料","text":"卷積在等資料","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"把卷積寫成矩陣乘法","text":"把卷積寫成矩陣乘法","depth":3},{"id":"視窗越大越快","text":"視窗越大越快","depth":3},{"id":"補上視窗之間的交流","text":"補上視窗之間的交流","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"偵測與分割","text":"偵測與分割","depth":3},{"id":"其他硬體","text":"其他硬體","depth":3},{"id":"限制","text":"限制","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/slak","relation":"referenced","topic":""},{"id":"reparameterization/unireplknet","relation":"referenced","topic":""}]},{"id":"reparameterization/replknet","category":"reparameterization","legacyPath":"reparameterization/2203-replknet","legacySlug":"2203-replknet","legacyPaths":[],"title":"RepLKNet","subtitle":"巨無霸卷積核","author":"Z. Yuan","paperDate":"2022-03-13","publishedOn":"2024-07-21","paperTitle":"Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs","paperUrl":"https://arxiv.org/abs/2203.06717","bibliography":{"title":"Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs","authors":["Xiaohan Ding","Xiangyu Zhang","Yizhuang Zhou","Jungong Han","Guiguang Ding","Jian Sun"],"url":"https://arxiv.org/abs/2203.06717","pdfUrl":"https://arxiv.org/pdf/2203.06717","codeUrl":null,"preprint":{"id":"2203.06717","submittedOn":"2022-03-13","url":"https://arxiv.org/abs/2203.06717"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://arxiv.org/abs/2203.06717"},"citation":"@misc{reparameterizationreplknet2022,\n  title = {{Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs}},\n  author = {Xiaohan Ding and Xiangyu Zhang and Yizhuang Zhou and Jungong Han and Guiguang Ding and Jian Sun},\n  year = {2022},\n  eprint = {2203.06717},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2203.06717}\n}\n","citationPath":"/content/papers/citations/reparameterization/replknet.c7cc2407aa19.bib"},"thumbnail":{"src":"/content/papers/assets/b85fa678b6e815b9d112b67d68443295a98c80478664ec8053c2e832d89574ca.jpg","width":1140,"height":972},"locale":"zh-hant","path":"/content/papers/zh-hant/reparameterization/replknet.16fd94d59060.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"大核卷積使用說明書","text":"大核卷積使用說明書","depth":2},{"id":"深度大核卷積仍然高效","text":"深度大核卷積仍然高效","depth":3},{"id":"大核卷積十分依賴殘差連接","text":"大核卷積十分依賴殘差連接","depth":3},{"id":"大核卷積需要重新參數化","text":"大核卷積需要重新參數化","depth":3},{"id":"大核卷積明顯提升下游任務性能","text":"大核卷積明顯提升下游任務性能","depth":3},{"id":"大核卷積對於小特徵圖的效果依然很好","text":"大核卷積對於小特徵圖的效果依然很好","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"網路架構","text":"網路架構","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"大核卷積評估","text":"大核卷積評估","depth":3},{"id":"imagenet-上的性能","text":"ImageNet 上的性能","depth":3},{"id":"大核-cnn-比深度小核模型有更大的-erf","text":"大核 CNN 比深度小核模型有更大的 ERF","depth":3},{"id":"形狀偏差","text":"形狀偏差","depth":3},{"id":"限制","text":"限制","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"comparison","topic":"大卷積核與簡化骨架"}]},{"id":"classic-cnns/slak","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"SLaK","subtitle":"卷積核 Pro Max","author":"Z. Yuan","paperDate":"2022-07-07","publishedOn":"2025-03-09","world":"external/slak","paperTitle":"More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity","paperUrl":"https://arxiv.org/abs/2207.03620","bibliography":{"title":"More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity","authors":["Shiwei Liu","Tianlong Chen","Xiaohan Chen","Xuxi Chen","Qiao Xiao","Boqian Wu","Tommi Kärkkäinen","Mykola Pechenizkiy","Decebal Mocanu","Zhangyang Wang"],"url":"https://arxiv.org/abs/2207.03620","pdfUrl":"https://arxiv.org/pdf/2207.03620v3","codeUrl":null,"preprint":{"id":"2207.03620","submittedOn":"2022-07-07","url":"https://arxiv.org/abs/2207.03620"},"publication":{"venue":"ICLR","year":2023,"kind":"conference","url":"https://iclr.cc/virtual/2023/poster/11529"},"citation":"@misc{classiccnnsslak2022,\n  title = {{More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity}},\n  author = {Shiwei Liu and Tianlong Chen and Xiaohan Chen and Xuxi Chen and Qiao Xiao and Boqian Wu and Tommi Kärkkäinen and Mykola Pechenizkiy and Decebal Mocanu and Zhangyang Wang},\n  year = {2022},\n  eprint = {2207.03620},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2207.03620}\n}\n","citationPath":"/content/papers/citations/classic-cnns/slak.e167e693df01.bib"},"thumbnail":{"src":"/content/papers/assets/b5b8ba279efe42831c7ec4e60395d0559da26774c8519a438783f49f4b814a85.png","width":1640,"height":660},"locale":"zh-hant","path":"/content/papers/zh-hant/classic-cnns/slak.9c5ac51ae21b.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"31-之後","text":"31 之後","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"第一步拆成兩根長條","text":"第一步：拆成兩根長條","depth":3},{"id":"第二步稀疏一點寬一點","text":"第二步：稀疏一點，寬一點","depth":3},{"id":"slak-的樣子","text":"SLaK 的樣子","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"下游任務","text":"下游任務","depth":3},{"id":"實際跑起來","text":"實際跑起來","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/resnext","relation":"referenced","topic":""}]},{"id":"reparameterization/unireplknet","category":"reparameterization","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"UniRepLKNet","subtitle":"大核跨界","author":"Z. Yuan","paperDate":"2023-11-27","publishedOn":"2025-12-14","world":"external/unireplknet","paperTitle":"UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition","paperUrl":"https://arxiv.org/abs/2311.15599","bibliography":{"title":"UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition","authors":["Xiaohan Ding","Yiyuan Zhang","Yixiao Ge","Sijie Zhao","Lin Song","Xiangyu Yue","Ying Shan"],"url":"https://arxiv.org/abs/2311.15599","pdfUrl":"https://arxiv.org/pdf/2311.15599v2","codeUrl":"https://github.com/AILab-CVC/UniRepLKNet","preprint":{"id":"2311.15599","submittedOn":"2023-11-27","url":"https://arxiv.org/abs/2311.15599"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2024/html/Ding_UniRepLKNet_A_Universal_Perception_Large-Kernel_ConvNet_for_Audio_Video_Point_CVPR_2024_paper.html"},"citation":"@misc{reparameterizationunireplknet2023,\n  title = {{UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition}},\n  author = {Xiaohan Ding and Yiyuan Zhang and Yixiao Ge and Sijie Zhao and Lin Song and Xiangyu Yue and Ying Shan},\n  year = {2023},\n  eprint = {2311.15599},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2311.15599}\n}\n","citationPath":"/content/papers/citations/reparameterization/unireplknet.df5ca5b9a0be.bib"},"thumbnail":{"src":"/content/papers/assets/8dc0f2f7d7d64f92bea8485e2e00428225297bb13404dee50fe50f28385d34c8.png","width":1680,"height":384},"locale":"zh-hant","path":"/content/papers/zh-hant/reparameterization/unireplknet.e887d30226f9.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"一層-33-做了三件事","text":"一層 3×3 做了三件事","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"基準架構","text":"基準架構","depth":3},{"id":"準則一加深用便宜的結構","text":"準則一：加深用便宜的結構","depth":3},{"id":"準則二用空洞小卷積重參數化","text":"準則二：用空洞小卷積重參數化","depth":3},{"id":"準則三卷積核大小看下游任務","text":"準則三：卷積核大小看下游任務","depth":3},{"id":"準則四加深時加的是小卷積核","text":"準則四：加深時，加的是小卷積核","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"影像","text":"影像","depth":3},{"id":"其他模態","text":"其他模態","depth":3},{"id":"代價","text":"代價","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"reparameterization/replknet","relation":"referenced","topic":""},{"id":"classic-cnns/slak","relation":"referenced","topic":""},{"id":"lightweight/senet","relation":"referenced","topic":""}]}],"text":"卷積核越做越大。ConvNeXt 用 7×7，RepLKNet 做到 31×31，SLaK 做到 51×51，UniRepLKNet 回頭選了 13×13。準確率一路往上，速度問題卻一直沒解決。\n\ndepthwise 卷積的 FLOPs 很少，卷積核放大之後卻慢得很快。這篇的作者量過，標準的 depthwise 卷積從 5×5 放大到 13×13，吞吐量掉了 71% 到 78%；放大到 27×27，掉了 92% 到 94%。不管批次大小、特徵圖多大，結果都差不多。\n\nRepLKNet 和 UniRepLKNet 為此寫了專用的 CUDA 運算子，論文裡把它叫做 LKA（Large Kernel Acceleration）。它在小特徵圖上有用，可是特徵圖一大反而更慢。在 224×224、批次 128 的設定下，LKA 的 7×7 比不加速的 5×5 慢了 80%，13×13 慢了 89%。偵測、分割、超解析度這些任務，偏偏都要處理大特徵圖。\n\n這篇來自合肥工業大學、Lingyang Industrial Internet 公司和日本高知工科大學。他們換了一個方向，不去最佳化卷積怎麼讀資料，而是讓卷積改用矩陣乘法來算。\n\n\n> [!TIP]\n> \n> 大卷積核這條路上的幾站，可以參考我們的筆記：\n> \n> - **[[22.03] RepLKNet：巨無霸卷積核](/blog/papers/reparameterization/replknet)**\n> - **[[22.07] SLaK：卷積核 Pro Max](/blog/papers/classic-cnns/slak)**\n> - **[[23.11] UniRepLKNet：大核跨界](/blog/papers/reparameterization/unireplknet)**\n\n\n## 定義問題\n\n### 卷積在等資料\n\n[![原論文 Figure 2：四個面板。(a) 5×5 depthwise 卷積，一個紅色的輸出位置 y 從周圍 5 列、每列 5 個格子收集數值，紅色細線從四面八方連到 y，註明每個輸出要從 5 個不連續的列收集 25 個值。(b) WBMM 的 4×4 視窗，8×8 的特徵圖被切成 W0 到 W3 四個不重疊的彩色區塊，註明每個視窗從一個連續的區塊讀 16 個值。(c) depthwise 卷積的記憶體存取，第 1 到第 5 列各有一段紅框，標示 fetch #1 到 fetch #5，下方列出 5 次獨立讀取、跨距等於寬度、工作集橫跨多條快取線、每個位元組 O(1) 次運算，屬於記憶體受限。(d) WBMM 的記憶體存取，四列中各取 4 個值的藍框，經過 window_partition 和 reshape 之後排成一條連續的 16 格，只要 fetch #1 一次，下方列出一次連續讀取、位址連續、視窗放得進 L1 快取、每個位元組 O(w²) 次運算，屬於計算受限。](/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png)](/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png)\n\n*原論文 Figure 2（Song 等人），取自 [**PDF 第 2 頁**](https://arxiv.org/pdf/2607.02097v2#page=2)。圖說未收錄。*\n\n問題出在記憶體存取。特徵圖在記憶體裡是一列接一列排的，depthwise 卷積的每個輸出，要從上下 $k$ 列各抓 $k$ 個值。這些值在記憶體裡不連續，同一列的 $k$ 個值相鄰，換到下一列就要跳過整整一行的寬度。\n\n卷積核越大，要跳的列越多，快取越裝不下。每讀進一個值只做一次乘加，GPU 的算力用不上，大部分時間在等資料。\n\n## 解決問題\n\n### 把卷積寫成矩陣乘法\n\n[![原論文 Figure 1：左半 (a) 是 depthwise 卷積，輸入 X 是一張大網格，中間一個紅色格子 (i, j) 周圍框出 7×7 的橘色範圍，和 7×7 的卷積核 W 做卷積，箭頭標示收集 7² = 49 個鄰居，得到輸出 y[i, j]。右半 (b) 是 WBMM，輸入 X 被切成四個顏色的 7×7 視窗；上方一條淺藍色的相對位置表 R，標示大小為 C × (2·7 − 1)² = C × 169，經過 index 箭頭組成一個 C × 49 × 49 的矩陣 M；輸入視窗經過標示 Batched MatMul 的綠色箭頭和 M 相乘，得到同樣切成四個視窗的輸出 Y。](/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png)](/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png)\n\n*原論文 Figure 1（Song 等人），取自 [**PDF 第 1 頁**](https://arxiv.org/pdf/2607.02097v2#page=1)。圖說未收錄。*\n\nWBMM（Windowed Batch Matrix Multiplication）分三步：\n\n1. **切視窗**：把特徵圖切成不重疊的 $w\\times w$ 視窗，例如 7×7，每個視窗攤平成 49 個值，在記憶體裡排成連續的一段。\n2. **查表組權重**：每個通道有一張相對位置表 $R$，大小是 $(2w-1)^2$，7×7 視窗就是 13×13 = 169 個參數。視窗裡第 $i$ 個位置對第 $j$ 個位置的權重，只看兩者的相對位置，去表裡查出來，組成一個 49×49 的矩陣 $M$。\n3. **批次矩陣乘法**：同一個通道的所有視窗共用同一個 $M$，所以把通道當成批次維度，一次 `bmm` 就算完所有視窗。\n\n權重只看相對位置，這正是卷積的定義。論文的定理 3.2 證明，在有界的特徵圖上，depthwise 卷積可以精確寫成依相對位置決定權重的矩陣乘法。套到每個視窗上，WBMM 就等於一個 13×13 的 depthwise 卷積，視窗外一律當成 0。我們照著寫一遍，和逐視窗做 13×13 卷積的結果比一比：\n\n```python\nimport torch\nimport torch.nn.functional as F\n\ntorch.manual_seed(0)\nC, H, W, w = 4, 14, 14, 7                       # 4 個通道、14×14 的特徵圖、7×7 的視窗\nd = w * w                                        # 每個視窗 49 個位置\nR = torch.randn(C, (2 * w - 1) ** 2)             # 相對位置表：每個通道 13×13 = 169 個參數\n\n# 相對位置索引：視窗裡第 i 個位置看第 j 個位置，該用表裡的哪一格\npos = torch.stack(torch.meshgrid(torch.arange(w), torch.arange(w), indexing=\"ij\")).flatten(1)  # 2 × 49\nrel = pos[:, None, :] - pos[:, :, None] + (w - 1)                                                # 2 × 49 × 49\nindex = rel[0] * (2 * w - 1) + rel[1]\n\nM = R[:, index]                                  # C × 49 × 49：查表組出權重矩陣\n\n# 把特徵圖切成不重疊的視窗，排成 (C, 視窗數, 49)\nx = torch.randn(1, C, H, W)\nnh, nw = H // w, W // w\nwindows = x.view(C, nh, w, nw, w).permute(0, 1, 3, 2, 4).reshape(C, nh * nw, d)\ny = torch.bmm(windows, M.transpose(1, 2))         # 每個通道一次批次矩陣乘法，算完所有視窗\ny = y.view(C, nh, nw, w, w).permute(0, 1, 3, 2, 4).reshape(1, C, H, W)\n\n# 對照：13×13 的 depthwise 卷積，一個視窗一個視窗算，視窗外補 0\nkernel = R.view(C, 1, 2 * w - 1, 2 * w - 1)\nref = torch.zeros_like(x)\nfor i in range(nh):\n    for j in range(nw):\n        patch = x[..., i * w:(i + 1) * w, j * w:(j + 1) * w]\n        ref[..., i * w:(i + 1) * w, j * w:(j + 1) * w] = F.conv2d(patch, kernel, padding=w - 1, groups=C)\n\nprint(\"權重矩陣：\", tuple(M.shape), \"，由\", R.shape[1], \"個參數查表組成\")\nprint(\"和視窗內 13×13 depthwise 卷積的最大差距：\", f\"{(y - ref).abs().max():.1e}\")\n```\n\n```text\n權重矩陣： (4, 49, 49) ，由 169 個參數查表組成\n和視窗內 13×13 depthwise 卷積的最大差距： 2.9e-06\n```\n\n兩種算法的結果一樣，差距只是浮點數誤差。差別在記憶體裡發生的事，卷積要一個一個去收集散落的鄰居，WBMM 讀一整塊連續的視窗，交給 GPU 最擅長的矩陣乘法。\n\n### 視窗越大越快\n\n這帶來一個反直覺的性質。depthwise 卷積的卷積核越大越慢，WBMM 卻是視窗越大越快。\n\n一個 $w\\times w$ 的視窗要讀 $w^2$ 個值，矩陣乘法要做 $w^4$ 次乘加，所以每讀一個值能做 $w^2$ 次運算。視窗越大，計算和讀取的比例越高，GPU 越有事做。權重矩陣 $M$ 雖然有 $w^4$ 個元素，但整批資料的所有視窗共用，讀一次就攤掉了。\n\n14×14 的視窗，每個輸出要做 196 次乘加，是 5×5 depthwise 卷積的將近 8 倍，看到的範圍也是 7.8 倍。論文的運算子測試在 A800 上用 FP32、256 個通道，WBMM 的 14×14 視窗和標準的 5×5 depthwise 卷積比：\n\n| 批次 | 14×14 特徵圖 | 28×28 | 56×56 | 112×112 | 224×224 |\n| ---: | ---: | ---: | ---: | ---: | ---: |\n| 4 | 0.45× | 0.37× | 1.41× | 1.64× | 2.02× |\n| 16 | 0.62× | 1.56× | 1.80× | 2.00× | 2.07× |\n| 64 | 1.86× | 1.85× | 2.01× | 2.01× | 2.03× |\n| 128 | 3.17× | 2.00× | 2.06× | 1.97× | 2.01× |\n| 256 | 3.72× | 2.02× | 2.02× | 1.95× | — |\n\n*表：摘自原論文 Table 1，數字是 WBMM 相對 5×5 depthwise 卷積的速度倍數。*\n\n特徵圖夠大、批次夠大時，WBMM 大約是 2 倍快。批次小、特徵圖又小的時候反而比較慢，批次 4、28×28 只有 0.37 倍。\n\n這和視窗注意力很不一樣。Swin 的視窗注意力每個樣本、每個視窗都要算一個依輸入變化的矩陣，再做 softmax；WBMM 的 $M$ 和輸入無關，整批共用一個，推論時還可以先算好存起來。\n\n### 補上視窗之間的交流\n\n視窗不重疊，資訊出不了視窗。作者在 WBMM 區塊之間穿插一般的 3×3 depthwise 卷積，讓相鄰視窗交換資訊。\n\n對分割這類需要多尺度的任務，還有一個階層式的視窗重參數化。訓練時同時有一個 14×14 的大視窗和四個 7×7 的小視窗，各自有相對位置表；推論前把小視窗的矩陣加進大視窗矩陣的對角區塊，合成一個矩陣，推論時沒有額外成本。這和 UniRepLKNet 把小卷積核合進大卷積核是同一個思路。\n\n整個模型以 UniRepLKNet 為基準，把大卷積核換成 WBMM，再調整各階段裡 WBMM 區塊和 3×3 區塊的排法。\n\n## 討論\n\n### ImageNet\n\n| 模型 | 每個 epoch 的訓練時間 | 訓練記憶體 | top-1 |\n| --- | ---: | ---: | ---: |\n| WBMM-T | **6:18** | **15.0 GB** | 83.2% |\n| UniRepLKNet-T（有 LKA） | 9:03 | 16.6 GB | 83.2% |\n| UniRepLKNet-T（沒有 LKA） | 11:03 | 16.6 GB | 83.2% |\n| WBMM-S | **8:35** | **20.4 GB** | 83.9% |\n| UniRepLKNet-S（有 LKA） | 12:10 | 22.3 GB | 83.9% |\n\n*表：摘自原論文 Table 5 的部分列。8 張 A800。*\n\n準確率完全一樣，訓練快了 1.4 倍以上，不用 LKA 的話差距更大。四個尺寸算下來，訓練加速是 1.31 到 1.88 倍，這是論文摘要裡的數字。\n\n不過這張表比的是 UniRepLKNet 整套，它訓練時還掛著好幾條空洞小卷積的重參數化分支。附錄有一張更公平的表，所有東西都一樣，只換空間運算子：\n\n| 空間運算子 | 每個 epoch | top-1 | ADE20K mIoU | 推論（張／秒） |\n| --- | ---: | ---: | ---: | ---: |\n| WBMM（7×7 視窗） | **6:20** | 83.0% | 48.3 | **1833** |\n| WBMM（14×14 階層式） | 6:31 | 83.2% | **48.8** | 1842 |\n| 13×13 depthwise ＋ LKA | 6:43 | 83.1% | 48.3 | 1662 |\n| 13×13 depthwise（沒有 LKA） | 9:11 | 83.1% | 48.3 | 1305 |\n| 13×13 depthwise ＋ 重參數化 ＋ LKA | 10:21 | 83.2% | 48.7 | 1662 |\n| SLaK 原版 | 22:03 | 83.3% | 47.4 | 549 |\n| 7×7 視窗注意力 | 8:38 | 83.3% | 48.1 | 1245 |\n\n*表：摘自原論文 Table 8 的部分列，全部是 Tiny 尺寸、同一套架構。*\n\n和一般的 13×13 depthwise 卷積加 LKA 比，WBMM 的訓練只快了 6%，推論快了 10%。前一張表那 1.4 倍，大半來自拿掉 UniRepLKNet 的多分支重參數化，10:21 變成 6:20。和沒有 LKA 的 depthwise 卷積比，WBMM 才是快 1.4 倍以上。\n\nWBMM 真正的價值在於，不用寫任何專用的 CUDA 運算子，就達到甚至超過 LKA 的速度。\n\n### 偵測與分割\n\n| 方法 | ADE20K mIoU（單尺度） | COCO box AP | COCO mask AP |\n| --- | ---: | ---: | ---: |\n| UniRepLKNet-T | 48.6 | 51.8 | 44.9 |\n| WBMM-T（7×7） | 48.3 | 51.6 | 44.8 |\n| WBMM-T（階層式） | **48.8** | **51.9** | **45.1** |\n\n*表：摘自原論文 Table 6 與 Table 7。ADE20K 用 UperNet，COCO 用 Cascade Mask R-CNN。*\n\n純 7×7 視窗的版本比 UniRepLKNet 低 0.1 到 0.3；加上階層式重參數化，才比它高 0.1 到 0.2，參數也多了 3M 到 4M。分割和偵測上兩者大致打平，WBMM 贏在速度。\n\n### 其他硬體\n\n作者也在其他裝置上量了速度。A800 上比有 LKA 的 UniRepLKNet 快 1.01 到 1.28 倍；Intel i7-13700K CPU 上沒有 LKA 可用，WBMM 靠 BLAS 函式庫快了 1.03 到 1.48 倍；Jetson Orin Nano 的 GPU 用 FP16 快了 1.19 到 3.12 倍，CPU 用 INT8 快了 1.33 到 2.44 倍。矩陣乘法是每種硬體都最佳化過的運算，這點在邊緣裝置上特別划算。\n\n### 限制\n\n作者在附錄列了幾個限制。權重矩陣 $M$ 的大小隨視窗邊長的四次方成長，14×14 視窗時一個 256 通道的層大約 37.5 MB，超過 28×28 就放不下了。視窗大小要看任務挑，純 14×14 在 ADE20K 上反而比純 7×7 差。權重和輸入無關，需要依內容調整的任務可能不夠用。\n\n## 結論\n\nWBMM 沒有改變模型看到什麼。在每個視窗裡，它就是一個 13×13 的 depthwise 卷積；它改的是資料在記憶體裡怎麼排，以及用哪一種運算子去算。卷積換成批次矩陣乘法之後，從等資料變成忙計算，視窗越大反而越快。\n\n讀表時要分清楚比的是誰。摘要裡那個 1.31 到 1.88 倍，是和整套 UniRepLKNet 比；只換運算子的話，和加了 LKA 的 depthwise 卷積比，訓練快的是 6%。它真正省下的，是那套專用的 CUDA 運算子。\n\n所以如果你的大卷積核模型卡在高解析度的速度，不妨先試這招。只要 PyTorch 的 `bmm`，上面示範程式的前半段就是全部的核心邏輯，連 CUDA 擴充都不用編。\n","imageSizes":{"/content/papers/assets/a830da4662436ed9b9ed1985cd4a866d805da44ae503db57f3af5ed78d1f3f63.png":{"width":1032,"height":864},"/content/papers/assets/5c20ec217f297d24ad8ce71ca79936d0599f7b460167e3bb89f52515ff1347ee.png":{"width":1017,"height":360}}}