{"notes":[{"id":"mamba/mambaout","category":"mamba","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"MambaOut","subtitle":"曼巴退场","author":"Z. Yuan","paperDate":"2024-05-13","publishedOn":"2025-11-18","world":"external/vim-2","paperTitle":"MambaOut: Do We Really Need Mamba for Vision?","paperUrl":"https://arxiv.org/abs/2405.07992","bibliography":{"title":"MambaOut: Do We Really Need Mamba for Vision?","authors":["Weihao Yu","Xinchao Wang"],"url":"https://arxiv.org/abs/2405.07992","pdfUrl":"https://arxiv.org/pdf/2405.07992v3","codeUrl":"https://github.com/yuweihao/MambaOut","preprint":{"id":"2405.07992","submittedOn":"2024-05-13","url":"https://arxiv.org/abs/2405.07992"},"publication":{"venue":"CVPR","year":2025,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2025/html/Yu_MambaOut_Do_We_Really_Need_Mamba_for_Vision_CVPR_2025_paper.html"},"citation":"@misc{mambamambaout2024,\n  title = {{MambaOut: Do We Really Need Mamba for Vision?}},\n  author = {Weihao Yu and Xinchao Wang},\n  year = {2024},\n  eprint = {2405.07992},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2405.07992}\n}\n","citationPath":"/content/papers/citations/mamba/mambaout.a7cd237c551a.bib"},"thumbnail":{"src":"/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png","width":1112,"height":456},"locale":"zh-hans","path":"/content/papers/zh-hans/mamba/mambaout.daa705ada440.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"mamba-适合什么样的任务","text":"Mamba 适合什么样的任务？","depth":3},{"id":"图像算长序列吗","text":"图像算长序列吗？","depth":3},{"id":"把-ssm-拿掉","text":"把 SSM 拿掉","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet-分类","text":"ImageNet 分类","depth":3},{"id":"检测与分割","text":"检测与分割","depth":3},{"id":"这把尺子的局限","text":"这把尺子的局限","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/poolformer","relation":"referenced","topic":""},{"id":"vision-transformers/caformer","relation":"referenced","topic":""},{"id":"mamba/vim","relation":"referenced","topic":""}]},{"id":"vision-transformers/poolformer","category":"vision-transformers","legacyPath":"vision-transformers/2111-poolformer","legacySlug":"2111-poolformer","legacyPaths":[],"title":"PoolFormer","subtitle":"你需要的是框架！","author":"Z. Yuan","paperDate":"2021-11-22","publishedOn":"2024-07-10","paperTitle":"MetaFormer Is Actually What You Need for Vision","paperUrl":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html","bibliography":{"title":"MetaFormer Is Actually What You Need for Vision","authors":["Weihao Yu","Mi Luo","Pan Zhou","Chenyang Si","Yichen Zhou","Xinchao Wang","Jiashi Feng","Shuicheng Yan"],"url":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html","pdfUrl":"https://openaccess.thecvf.com/content/CVPR2022/papers/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.pdf","codeUrl":"https://github.com/sail-sg/poolformer","preprint":{"id":"2111.11418","submittedOn":"2021-11-22","url":"https://arxiv.org/abs/2111.11418"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html"},"citation":"@misc{visiontransformerspoolformer2021,\n  title = {{MetaFormer Is Actually What You Need for Vision}},\n  author = {Weihao Yu and Mi Luo and Pan Zhou and Chenyang Si and Yichen Zhou and Xinchao Wang and Jiashi Feng and Shuicheng Yan},\n  year = {2021},\n  eprint = {2111.11418},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2111.11418}\n}\n","citationPath":"/content/papers/citations/vision-transformers/poolformer.f2f96629d283.bib"},"thumbnail":{"src":"/content/papers/assets/3cb141e9bb1247410c8214276b21b28ed1d536060d470406f1bfb0c88fe68df2.jpg","width":1498,"height":548},"locale":"zh-hans","path":"/content/papers/zh-hans/vision-transformers/poolformer.1460bbffaf4a.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"input-embedding","text":"Input Embedding","depth":3},{"id":"token-mixer","text":"Token-Mixer","depth":3},{"id":"channel-mlp","text":"Channel MLP","depth":3},{"id":"或许-token-mixer-不是重点","text":"或许 Token-Mixer 不是重点？","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"模型配置","text":"模型配置","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet-实验","text":"ImageNet 实验","depth":3},{"id":"消融实验","text":"消融实验","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/mlp-mixer","relation":"background","topic":"把骨架与 Token 混合方式分开看"},{"id":"vision-transformers/caformer","relation":"next","topic":"把骨架与 Token 混合方式分开看"}]},{"id":"vision-transformers/caformer","category":"vision-transformers","legacyPath":"vision-transformers/2210-caformer","legacySlug":"2210-caformer","legacyPaths":[],"title":"CAFormer","subtitle":"MetaFormer 使用说明书","author":"Z. Yuan","paperDate":"2022-10-24","publishedOn":"2024-07-11","paperTitle":"MetaFormer Baselines for Vision","paperUrl":"https://doi.org/10.1109/TPAMI.2023.3329173","bibliography":{"title":"MetaFormer Baselines for Vision","authors":["Weihao Yu","Chenyang Si","Pan Zhou","Mi Luo","Yichen Zhou","Jiashi Feng","Shuicheng Yan","Xinchao Wang"],"url":"https://doi.org/10.1109/TPAMI.2023.3329173","pdfUrl":"https://arxiv.org/pdf/2210.13452v4","codeUrl":"https://github.com/sail-sg/metaformer","preprint":{"id":"2210.13452","submittedOn":"2022-10-24","url":"https://arxiv.org/abs/2210.13452"},"publication":{"venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","shortVenue":"IEEE TPAMI","year":2024,"kind":"journal","url":"https://doi.org/10.1109/TPAMI.2023.3329173"},"citation":"@misc{visiontransformerscaformer2022,\n  title = {{MetaFormer Baselines for Vision}},\n  author = {Weihao Yu and Chenyang Si and Pan Zhou and Mi Luo and Yichen Zhou and Jiashi Feng and Shuicheng Yan and Xinchao Wang},\n  year = {2022},\n  eprint = {2210.13452},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2210.13452}\n}\n","citationPath":"/content/papers/citations/vision-transformers/caformer.aa84265d674f.bib"},"thumbnail":{"src":"/content/papers/assets/6ccbfbc74d7ee0be5bc2f37dd6cdd936bf9ed5cf9412cdab0f7dcc2c77a6d9ec.jpg","width":1224,"height":1004},"locale":"zh-hans","path":"/content/papers/zh-hans/vision-transformers/caformer.be913634f4a3.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"starrelu","text":"StarReLU","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"使用基本的-token-mixer","text":"使用基本的 Token Mixer","depth":3},{"id":"使用常见的-token-mixer","text":"使用常见的 Token Mixer","depth":3},{"id":"消融实验","text":"消融实验","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/poolformer","relation":"background","topic":"把骨架与 Token 混合方式分开看"},{"id":"vision-transformers/convmixer","relation":"comparison","topic":"用卷积混合图像分块"},{"id":"classic-cnns/convnext","relation":"comparison","topic":"用卷积混合图像分块"}]},{"id":"mamba/vim","category":"mamba","legacyPath":"mamba/2401-vim","legacySlug":"2401-vim","legacyPaths":[],"title":"Vim","subtitle":"曼巴的视线","author":"Z. Yuan","paperDate":"2024-01-17","publishedOn":"2025-01-08","paperTitle":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","paperUrl":"https://arxiv.org/abs/2401.09417","bibliography":{"title":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","authors":["Lianghui Zhu","Bencheng Liao","Qian Zhang","Xinlong Wang","Wenyu Liu","Xinggang Wang"],"url":"https://arxiv.org/abs/2401.09417","pdfUrl":"https://arxiv.org/pdf/2401.09417","codeUrl":"https://github.com/hustvl/Vim","preprint":{"id":"2401.09417","submittedOn":"2024-01-17","url":"https://arxiv.org/abs/2401.09417"},"publication":{"venue":"ICML","year":2024,"kind":"conference","url":"https://arxiv.org/abs/2401.09417"},"citation":"@misc{mambavim2024,\n  title = {{Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model}},\n  author = {Lianghui Zhu and Bencheng Liao and Qian Zhang and Xinlong Wang and Wenyu Liu and Xinggang Wang},\n  year = {2024},\n  eprint = {2401.09417},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2401.09417}\n}\n","citationPath":"/content/papers/citations/mamba/vim.04f7a0c3f9ed.bib"},"thumbnail":{"src":"/content/papers/assets/f6f1543dc8b09bec8d718b356195b440e445961b944f10cc2cca7bbae4ba11e1.jpg","width":1616,"height":440},"locale":"zh-hans","path":"/content/papers/zh-hans/mamba/vim.4c8d8da6452b.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"模型架构","text":"模型架构","depth":3},{"id":"vision-mamba-block","text":"Vision Mamba Block","depth":3},{"id":"参数设定","text":"参数设定","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"图像分类的性能","text":"图像分类的性能","depth":3},{"id":"高分辨率下性能比较","text":"高分辨率下性能比较","depth":3},{"id":"图像分割的性能","text":"图像分割的性能","depth":3},{"id":"物体检测的性能","text":"物体检测的性能","depth":3},{"id":"消融实验---双向设计","text":"消融实验 - 双向设计","depth":3},{"id":"消融实验---分类策略","text":"消融实验 - 分类策略","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"mamba/mamba","relation":"background","topic":"从状态空间记忆到视觉序列"}]}],"text":"这篇论文的扉页上写着：In memory of Kobe Bryant。\n\n下面引用了 Kobe 2016 年 NBA 告别演讲的结尾：「What can I say, Mamba out.」Black Mamba 是 Kobe 的绰号，「Mamba out」就是曼巴退场。\n\n我们读 Mamba 的时候说过，那又是一个海量新论文的开始。视觉领域的反应确实很快：2024 年 1 月，Vision Mamba、VMamba 相继出现，接着是 LocalMamba、PlainMamba、EfficientVMamba，把 Mamba 放进分类、检测、分割的各种骨干网络。\n\n不到半年，新加坡国立大学的 Weihao Yu 与 Xinchao Wang 写了这篇，标题直接发问：视觉真的需要 Mamba 吗？\n\n如果你读过我们的 PoolFormer 与 CAFormer 笔记，应该认得第一作者。他一直在问同一类问题：模型表现好，到底是哪个零件的功劳？\n\n## 定义问题\n\n作者的出发点是一个观察：这些视觉 Mamba 模型的成绩，和当时最好的卷积或注意力模型相比，往往不够亮眼。\n\n那么 Mamba 适合做什么？视觉任务又是否符合？作者先从概念上回答这两个问题，再用一组刻意去掉 SSM 的模型来验证。\n\n## 解决问题\n\n### Mamba 适合什么样的任务？\n\n[![两栏示意图。左栏 Causal attention：第一步旧记忆为空，加入 x1 后新记忆存了 k1、v1；第二步新记忆变成 k1 v1、k2 v2 两组；第三步变成三组，记忆随步数变长。右栏 RNN-like：每一步都把旧的隐藏状态 h 和新输入融合成一个同样大小的新状态，h0 到 h1、h1 到 h2、h2 到 h3](/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png)](/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png)\n\n*原论文 Figure 2，取自 [**v3 PDF 第 4 页**](https://arxiv.org/pdf/2405.07992v3#page=4)。左边是因果注意力，右边是 RNN 类模型。*\n\n这张图把两种记忆方式画得很清楚。\n\n左边的因果注意力，每读一个 token，就把它的 key 和 value 加进记忆。每个位置的 key、value 都留着，代价是记忆越来越长：第三步要拿新输入和三组 key、value 一起算，第一千步就是一千组。\n\n右边的 RNN 类模型，包括 Mamba 的 SSM，每一步都把旧状态和新输入融合成一个同样大小的新状态。融合的成本固定，但状态大小有限，旧信息一定会被压缩或丢弃。\n\n作者由此推出 Mamba 的两个适用条件：\n\n1. **序列要足够长**。序列短时，注意力的成本还不高，又保留着每个位置的 key、value，作者认为 Mamba 很难占到便宜；序列足够长，注意力的平方成本才会成为负担。\n2. **任务需要因果的信息流**。SSM 的递归天生只能看过去和现在，适合一个字接一个字生成的任务。\n\n第二点需要多说一句。作者把 token 混合分成两种模式：**因果模式**，第 $t$ 个 token 只能看第 1 到第 $t$ 个；**完全可见模式**，每个 token 都能看到全部。注意力两种都能做，加上掩码就是因果模式。单向 RNN 只能沿着读取方向累积信息；要让每个位置看到上下文，就得另外加一个反方向的分支再合并。合并后每个位置确实用得上整段输入，但信息仍要经过两份固定大小的状态，和注意力的直接互看不一样。作者把这种情况也归为因果模式的延伸，认为它不如完全可见的注意力自然。\n\n图像识别是理解任务，整张图一次就能看到，不需要因果限制。作者做了一个小实验：把 ViT 的注意力改成因果模式，ImageNet 的准确率就下降了（原论文 Figure 3b）。\n\n### 图像算长序列吗？\n\n第一个条件需要一把尺子。作者用 Transformer 块的计算量来衡量：输入有 $L$ 个 token、每个 $D$ 维，MLP 扩张 4 倍时，\n\n$$\n\\text{FLOPs}=24D^2L+4DL^2,\\qquad r_L=\\frac{4DL^2}{24D^2L}=\\frac{L}{6D}\n$$\n\n$r_L$ 大于 1，也就是 $L>6D$ 时，与长度平方相关的那一项就超过线性项，可以算作长序列。\n\n我们拿论文的三个任务算一次，patch 都是 16×16：\n\n```python\nD = {\"ViT-S\": 384, \"ViT-B\": 768}\ntasks = {\"ImageNet 224×224\": (224, 224), \"COCO 800×1280\": (800, 1280), \"ADE20K 512×2048\": (512, 2048)}\nfor name, (h, w) in tasks.items():\n    L = (h // 16) * (w // 16)\n    ratios = \"，\".join(f\"{k} 的 L/6D = {L / (6 * d):.2f}\" for k, d in D.items())\n    print(f\"{name}: {L:,} 个 token；{ratios}\")\n```\n\n```text\nImageNet 224×224: 196 个 token；ViT-S 的 L/6D = 0.09，ViT-B 的 L/6D = 0.04\nCOCO 800×1280: 4,000 个 token；ViT-S 的 L/6D = 1.74，ViT-B 的 L/6D = 0.87\nADE20K 512×2048: 4,096 个 token；ViT-S 的 L/6D = 1.78，ViT-B 的 L/6D = 0.89\n```\n\nImageNet 分类只有 196 个 token，离门槛差了十倍以上。COCO 检测与 ADE20K 分割约 4K 个 token，对 ViT-S 已经过了门槛，对 ViT-B 也接近了。\n\n于是作者提出了两个假设：\n\n- **假设一**：ImageNet 分类既不长，也不需要因果，用不到 SSM。\n- **假设二**：检测与分割虽然不需要因果，但序列足够长，SSM 也许还有帮助。\n\n### 把 SSM 拿掉\n\n[![左图是两种模块的结构：Gated CNN 模块的输入经过两个 Linear 分成两支，一支经过 Conv，另一支经过激活函数，两支相乘后再经过 Linear；Mamba 模块相同，只是 Conv 之后多了激活函数与 SSM。右图是 ImageNet Top-1 准确率对计算量的气泡图，MambaOut 的曲线位于 VMamba、PlainMamba 与 Vision Mamba 之上](/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png)](/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png)\n\n*原论文 Figure 1，取自 [**v3 PDF 第 1 页**](https://arxiv.org/pdf/2405.07992v3#page=1)。(a) 省略了归一化与残差连接；(b) 气泡大小代表参数量。*\n\n看左图，两个模块几乎一模一样。作者把它们写成同一个式子：\n\n$$\nY=\\big(\\text{TokenMixer}(X'W_1)\\odot\\sigma(X'W_2)\\big)W_3+X\n$$\n\n$X'$ 是归一化后的输入，一支经过 token mixer，另一支经过激活函数作为门控，两支逐元素相乘，再投影回原来的维度。差别只在 token mixer：\n\n$$\n\\text{TokenMixer}_{\\text{Gated CNN}}(Z)=\\text{Conv}(Z),\\qquad\n\\text{TokenMixer}_{\\text{Mamba}}(Z)=\\text{SSM}\\big(\\sigma(\\text{Conv}(Z))\\big)\n$$\n\nMamba 模块就是在 Gated CNN 的卷积后面多接一个 SSM。\n\n所以，把 Mamba 模块的 SSM 拿掉，剩下的就是 Gated CNN。作者用它堆出一系列模型，取名为 **MambaOut**：\n\n- token mixer 用 7×7 的 depthwise 卷积，沿用 ConvNeXt 的做法；为了实际速度，只对部分通道做卷积，沿用 InceptionNeXt 的做法。\n- 像 ResNet 一样分四个阶段，阶段之间做下采样。\n- 从最小的 Femto 到 Base，共四种尺寸。\n\n这个设计是一条基线：不含 SSM，看看能做到多好。需要说明的是，它和各个视觉 Mamba 之间不只差一个 SSM，卷积的写法、模块设计和各阶段配置都不同。所以就算 MambaOut 更好，严格来说证明的也是「不用 SSM 也能达到这个水平」；SSM 本身贡献了多少，要在同一个骨干网络上把它拿掉才量得出来。\n\n## 讨论\n\n### ImageNet 分类\n\n训练沿用 DeiT 的配方，不用蒸馏，输入 224×224。挑几组参数量相近的来比较：\n\n| 模型 | 参数（M） | MACs（G） | Top-1（%） |\n| --- | ---: | ---: | ---: |\n| Vim-Ti | 7 | 1.5 | 76.1 |\n| EfficientVMamba-S | 11 | 1.3 | 78.7 |\n| **MambaOut-Femto** | 7 | 1.2 | **78.9** |\n| VMamba-T | 22 | 5.6 | 82.2 |\n| LocalVMamba-T | 26 | 5.7 | 82.7 |\n| **MambaOut-Tiny** | 27 | 4.5 | **82.7** |\n| LocalVMamba-S | 50 | 11.4 | 83.7 |\n| VMambaV9-S | 50 | 8.7 | 83.6 |\n| **MambaOut-Small** | 48 | 9.0 | **84.1** |\n| VMambaV9-B | 89 | 15.4 | 83.9 |\n| **MambaOut-Base** | 85 | 15.8 | **84.2** |\n\n*表：原论文 Table 1 节选。*\n\n每个规模下，MambaOut 都追平或超过了带 SSM 的视觉 Mamba。以 Small 为例，MambaOut 比 LocalVMamba-S 高 0.4 个百分点，计算量只有后者的 79%；不过和 VMambaV9 相比，MambaOut 的 MACs 略高一些。\n\n原论文 Table 1 里还有一组值得对照的数字。CAFormer-M36 用 56M 参数、13.2G MACs 拿到 85.2%；参数更多的 VMambaV9-B（89M、15.4G）是 83.9%。也就是说，评估 SSM 值不值得用，最强的卷积与注意力骨干也得一起放进来比较。\n\n\n> [!TIP]\n> \n> CAFormer 与 MetaFormer 这条研究线，也出自同一位作者：\n> \n> - **[[21.11] PoolFormer：你需要的是框架！](/blog/papers/vision-transformers/poolformer)**\n> - **[[22.10] CAFormer：MetaFormer 使用说明书](/blog/papers/vision-transformers/caformer)**\n\n\n### 检测与分割\n\n换成长序列的任务，结果就反过来了。COCO 上用 Mask R-CNN、1× 训练策略：\n\n| 骨干网络 | AP（框） | AP（掩码） |\n| --- | ---: | ---: |\n| MambaOut-Tiny | 45.1 | 41.0 |\n| VMamba-T | 46.5 | 42.1 |\n| LocalVMamba-T | 46.7 | 42.2 |\n| VMambaV9-T | 47.4 | 42.7 |\n\n*表：原论文 Table 2 节选。*\n\nMambaOut-Tiny 比 VMamba-T 低了 1.4 个框 AP、1.1 个掩码 AP。ADE20K 语义分割也类似：MambaOut-Tiny 的单尺度 mIoU 为 47.4，和 VMamba-T 的 47.3 差不多，但低于 LocalVMamba-T 的 47.9 与 VMambaV9-T 的 48.3。\n\n作者把这解读为对假设二的支持：序列足够长时，SSM 可能帮上了忙。和分类一样，这里比较的也是不同的整体架构，这个结果更适合当作继续研究的线索。\n\n不过他们也没有替视觉 Mamba 说太多好话。在 COCO（Table 2，Mask R-CNN）与 ADE20K（Table 3，UperNet）两张表上，卷积加注意力的 TransNeXt 都领先同级别的视觉 Mamba；例如 ADE20K 单尺度 mIoU，TransNeXt-Tiny 是 51.1。\n\n### 这把尺子的局限\n\n$L>6D$ 这个门槛，衡量的是单个 Transformer 块的计算量什么时候被平方项主导。它是一个方便的判断依据，但不能拿来证明某个任务「一定需要」线性时间的模型。本文另外补充一点：检测与分割的骨干网络也常用窗口注意力或层级式下采样来降低成本，平方项的负担未必真的会出现。\n\n另外，假设二的证据比假设一弱一些。上面 Tiny 组的检测差距是 1.4 到 2.3 个框 AP、1.1 到 1.7 个掩码 AP，分割上有些视觉 Mamba 也没有赢过 MambaOut。这篇对「ImageNet 分类不需要 SSM」的支持很充分；至于「长序列视觉任务需要 SSM」，更接近「值得继续研究」。\n\n## 结论\n\n这篇论文用一条简单的基线，回答了标题里的问题：\n\n1. **作者认为 Mamba 适合长序列、因果式的任务**。固定大小的记忆在短序列上占不到便宜，单向递归也只沿着读取方向累积信息。\n2. **ImageNet 分类两个条件都不符合**。不含 SSM 的 MambaOut 在表中各个规模上都追平或胜过视觉 Mamba；MACs 的高低则因比较对象而异。\n3. **检测与分割是长序列，SSM 可能有一点帮助**。MambaOut 在这里输给了最好的视觉 Mamba，但所有视觉 Mamba 又都输给了更强的卷积与注意力混合模型。\n\nVision Mamba 的火苗才刚要熊熊燃起，就这样被掐灭了。\n\n唉，世事无常。\n\n\n> [!TIP]\n> \n> 把 Mamba 搬进视觉领域的第一批工作之一，可以参考：\n> \n> - **[[24.01] Vim：曼巴的视线](/blog/papers/mamba/vim)**\n","imageSizes":{"/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png":{"width":983,"height":619},"/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png":{"width":1112,"height":456}}}