{"notes":[{"id":"mamba/mambaout","category":"mamba","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"MambaOut","subtitle":"曼巴退場","author":"Z. Yuan","paperDate":"2024-05-13","publishedOn":"2025-11-18","world":"external/vim-2","paperTitle":"MambaOut: Do We Really Need Mamba for Vision?","paperUrl":"https://arxiv.org/abs/2405.07992","bibliography":{"title":"MambaOut: Do We Really Need Mamba for Vision?","authors":["Weihao Yu","Xinchao Wang"],"url":"https://arxiv.org/abs/2405.07992","pdfUrl":"https://arxiv.org/pdf/2405.07992v3","codeUrl":"https://github.com/yuweihao/MambaOut","preprint":{"id":"2405.07992","submittedOn":"2024-05-13","url":"https://arxiv.org/abs/2405.07992"},"publication":{"venue":"CVPR","year":2025,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2025/html/Yu_MambaOut_Do_We_Really_Need_Mamba_for_Vision_CVPR_2025_paper.html"},"citation":"@misc{mambamambaout2024,\n  title = {{MambaOut: Do We Really Need Mamba for Vision?}},\n  author = {Weihao Yu and Xinchao Wang},\n  year = {2024},\n  eprint = {2405.07992},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2405.07992}\n}\n","citationPath":"/content/papers/citations/mamba/mambaout.a7cd237c551a.bib"},"thumbnail":{"src":"/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png","width":1112,"height":456},"locale":"zh-hant","path":"/content/papers/zh-hant/mamba/mambaout.b34bcc976307.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"mamba-適合什麼樣的任務","text":"Mamba 適合什麼樣的任務？","depth":3},{"id":"影像算長序列嗎","text":"影像算長序列嗎？","depth":3},{"id":"把-ssm-拿掉","text":"把 SSM 拿掉","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet-分類","text":"ImageNet 分類","depth":3},{"id":"偵測與分割","text":"偵測與分割","depth":3},{"id":"這把尺的限制","text":"這把尺的限制","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/poolformer","relation":"referenced","topic":""},{"id":"vision-transformers/caformer","relation":"referenced","topic":""},{"id":"mamba/vim","relation":"referenced","topic":""}]},{"id":"vision-transformers/poolformer","category":"vision-transformers","legacyPath":"vision-transformers/2111-poolformer","legacySlug":"2111-poolformer","legacyPaths":[],"title":"PoolFormer","subtitle":"你需要的是框架！","author":"Z. Yuan","paperDate":"2021-11-22","publishedOn":"2024-07-10","paperTitle":"MetaFormer Is Actually What You Need for Vision","paperUrl":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html","bibliography":{"title":"MetaFormer Is Actually What You Need for Vision","authors":["Weihao Yu","Mi Luo","Pan Zhou","Chenyang Si","Yichen Zhou","Xinchao Wang","Jiashi Feng","Shuicheng Yan"],"url":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html","pdfUrl":"https://openaccess.thecvf.com/content/CVPR2022/papers/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.pdf","codeUrl":"https://github.com/sail-sg/poolformer","preprint":{"id":"2111.11418","submittedOn":"2021-11-22","url":"https://arxiv.org/abs/2111.11418"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2022/html/Yu_MetaFormer_Is_Actually_What_You_Need_for_Vision_CVPR_2022_paper.html"},"citation":"@misc{visiontransformerspoolformer2021,\n  title = {{MetaFormer Is Actually What You Need for Vision}},\n  author = {Weihao Yu and Mi Luo and Pan Zhou and Chenyang Si and Yichen Zhou and Xinchao Wang and Jiashi Feng and Shuicheng Yan},\n  year = {2021},\n  eprint = {2111.11418},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2111.11418}\n}\n","citationPath":"/content/papers/citations/vision-transformers/poolformer.f2f96629d283.bib"},"thumbnail":{"src":"/content/papers/assets/3cb141e9bb1247410c8214276b21b28ed1d536060d470406f1bfb0c88fe68df2.jpg","width":1498,"height":548},"locale":"zh-hant","path":"/content/papers/zh-hant/vision-transformers/poolformer.cc4d5c111221.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"input-embedding","text":"Input Embedding","depth":3},{"id":"token-mixer","text":"Token-Mixer","depth":3},{"id":"channel-mlp","text":"Channel MLP","depth":3},{"id":"或許-token-mixer-不是重點","text":"或許 Token-Mixer 不是重點？","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"模型配置","text":"模型配置","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"imagenet-實驗","text":"ImageNet 實驗","depth":3},{"id":"消融實驗","text":"消融實驗","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/mlp-mixer","relation":"background","topic":"把骨架與 Token 混合方式分開看"},{"id":"vision-transformers/caformer","relation":"next","topic":"把骨架與 Token 混合方式分開看"}]},{"id":"vision-transformers/caformer","category":"vision-transformers","legacyPath":"vision-transformers/2210-caformer","legacySlug":"2210-caformer","legacyPaths":[],"title":"CAFormer","subtitle":"MetaFormer 使用說明書","author":"Z. Yuan","paperDate":"2022-10-24","publishedOn":"2024-07-11","paperTitle":"MetaFormer Baselines for Vision","paperUrl":"https://doi.org/10.1109/TPAMI.2023.3329173","bibliography":{"title":"MetaFormer Baselines for Vision","authors":["Weihao Yu","Chenyang Si","Pan Zhou","Mi Luo","Yichen Zhou","Jiashi Feng","Shuicheng Yan","Xinchao Wang"],"url":"https://doi.org/10.1109/TPAMI.2023.3329173","pdfUrl":"https://arxiv.org/pdf/2210.13452v4","codeUrl":"https://github.com/sail-sg/metaformer","preprint":{"id":"2210.13452","submittedOn":"2022-10-24","url":"https://arxiv.org/abs/2210.13452"},"publication":{"venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","shortVenue":"IEEE TPAMI","year":2024,"kind":"journal","url":"https://doi.org/10.1109/TPAMI.2023.3329173"},"citation":"@misc{visiontransformerscaformer2022,\n  title = {{MetaFormer Baselines for Vision}},\n  author = {Weihao Yu and Chenyang Si and Pan Zhou and Mi Luo and Yichen Zhou and Jiashi Feng and Shuicheng Yan and Xinchao Wang},\n  year = {2022},\n  eprint = {2210.13452},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2210.13452}\n}\n","citationPath":"/content/papers/citations/vision-transformers/caformer.aa84265d674f.bib"},"thumbnail":{"src":"/content/papers/assets/6ccbfbc74d7ee0be5bc2f37dd6cdd936bf9ed5cf9412cdab0f7dcc2c77a6d9ec.jpg","width":1224,"height":1004},"locale":"zh-hant","path":"/content/papers/zh-hant/vision-transformers/caformer.dba87ff476ce.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"starrelu","text":"StarReLU","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"使用基本的-token-mixer","text":"使用基本的 Token Mixer","depth":3},{"id":"使用常見的-token-mixer","text":"使用常見的 Token Mixer","depth":3},{"id":"消融實驗","text":"消融實驗","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/poolformer","relation":"background","topic":"把骨架與 Token 混合方式分開看"},{"id":"vision-transformers/convmixer","relation":"comparison","topic":"用卷積混合影像切塊"},{"id":"classic-cnns/convnext","relation":"comparison","topic":"用卷積混合影像切塊"}]},{"id":"mamba/vim","category":"mamba","legacyPath":"mamba/2401-vim","legacySlug":"2401-vim","legacyPaths":[],"title":"Vim","subtitle":"曼巴的視線","author":"Z. Yuan","paperDate":"2024-01-17","publishedOn":"2025-01-08","paperTitle":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","paperUrl":"https://arxiv.org/abs/2401.09417","bibliography":{"title":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","authors":["Lianghui Zhu","Bencheng Liao","Qian Zhang","Xinlong Wang","Wenyu Liu","Xinggang Wang"],"url":"https://arxiv.org/abs/2401.09417","pdfUrl":"https://arxiv.org/pdf/2401.09417","codeUrl":"https://github.com/hustvl/Vim","preprint":{"id":"2401.09417","submittedOn":"2024-01-17","url":"https://arxiv.org/abs/2401.09417"},"publication":{"venue":"ICML","year":2024,"kind":"conference","url":"https://arxiv.org/abs/2401.09417"},"citation":"@misc{mambavim2024,\n  title = {{Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model}},\n  author = {Lianghui Zhu and Bencheng Liao and Qian Zhang and Xinlong Wang and Wenyu Liu and Xinggang Wang},\n  year = {2024},\n  eprint = {2401.09417},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2401.09417}\n}\n","citationPath":"/content/papers/citations/mamba/vim.04f7a0c3f9ed.bib"},"thumbnail":{"src":"/content/papers/assets/f6f1543dc8b09bec8d718b356195b440e445961b944f10cc2cca7bbae4ba11e1.jpg","width":1616,"height":440},"locale":"zh-hant","path":"/content/papers/zh-hant/mamba/vim.5c1a46431a8c.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"vision-mamba-block","text":"Vision Mamba Block","depth":3},{"id":"參數設定","text":"參數設定","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"影像分類的性能","text":"影像分類的性能","depth":3},{"id":"高解析度下性能比較","text":"高解析度下性能比較","depth":3},{"id":"影像分割的性能","text":"影像分割的性能","depth":3},{"id":"物件偵測的性能","text":"物件偵測的性能","depth":3},{"id":"消融實驗---雙向設計","text":"消融實驗 - 雙向設計","depth":3},{"id":"消融實驗---分類策略","text":"消融實驗 - 分類策略","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"mamba/mamba","relation":"background","topic":"從狀態空間記憶到視覺序列"}]}],"text":"這篇論文的扉頁寫著：In memory of Kobe Bryant。\n\n下面引了一句 Kobe 在 2016 年 NBA 告別演說的結尾：「What can I say, Mamba out.」Black Mamba 是 Kobe 的綽號，「Mamba out」就是曼巴退場。\n\n我們讀 Mamba 的時候說過，那又是一個海量新論文的開始。視覺領域的反應確實很快：2024 年 1 月 Vision Mamba、VMamba 先後出現，接著是 LocalMamba、PlainMamba、EfficientVMamba，把 Mamba 放進分類、偵測、分割的各種骨幹。\n\n半年不到，新加坡國立大學的 Weihao Yu 與 Xinchao Wang 寫了這篇，題目直接問：視覺真的需要 Mamba 嗎？\n\n如果你讀過我們的 PoolFormer 與 CAFormer 筆記，會認得第一作者。他過去一直在問同一類問題：模型表現好，到底是哪個零件的功勞？\n\n## 定義問題\n\n作者的出發點是一個觀察：這些視覺 Mamba 模型的成績，和當時最好的卷積或注意力模型相比，常常不夠亮眼。\n\n那 Mamba 適合做什麼？視覺任務又符合嗎？作者先從概念上回答這兩個問題，再用一組刻意拿掉 SSM 的模型驗證。\n\n## 解決問題\n\n### Mamba 適合什麼樣的任務？\n\n[![兩欄示意圖。左欄 Causal attention：第一步舊記憶為空，加入 x1 後新記憶存了 k1、v1；第二步新記憶變成 k1 v1、k2 v2 兩組；第三步變成三組，記憶隨步數變長。右欄 RNN-like：每一步都把舊的隱藏狀態 h 和新輸入融合成一個同樣大小的新狀態，h0 到 h1、h1 到 h2、h2 到 h3](/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png)](/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png)\n\n*原論文 Figure 2，取自 [**v3 PDF 第 4 頁**](https://arxiv.org/pdf/2405.07992v3#page=4)。左邊是因果注意力，右邊是 RNN 類的模型。*\n\n這張圖把兩種記憶方式畫得很清楚。\n\n左邊的因果注意力，每讀一個 token，就把它的 key 和 value 加進記憶。每個位置的 key、value 都留著，代價是越來越長：第三步要拿新輸入和三組 key、value 一起算，第一千步就是一千組。\n\n右邊的 RNN 類模型，包括 Mamba 的 SSM，每一步都把舊狀態和新輸入融合成一個同樣大小的新狀態。融合的成本固定，但狀態大小有限，舊資訊一定會被壓縮或丟掉。\n\n作者從這裡推出 Mamba 的兩個適用條件：\n\n1. **序列要夠長**。短序列時，注意力的成本還不高，又保留著每個位置的 key、value，作者認為 Mamba 很難占到便宜；序列夠長，注意力的平方成本才會變成負擔。\n2. **任務需要因果的資訊流**。SSM 的遞迴天生只能看過去和現在，適合一個字接一個字生成的任務。\n\n第二點需要多說一句。作者把 token 混合分成兩種模式：**因果模式**，第 $t$ 個 token 只能看第 1 到 $t$ 個；**完全可見模式**，每個 token 都能看到全部。注意力兩種都能做，加上遮罩就是因果模式。單向的 RNN 只能沿著讀的方向累積資訊；要讓每個位置看到前後文，就得另外加一個反方向的分支再合併。合併後每個位置確實用得到整段輸入，但資訊仍要經過兩份固定大小的狀態，和注意力直接互看不一樣。作者把這種情況也歸在因果模式的延伸，認為它不如完全可見的注意力自然。\n\n影像辨識是理解任務，整張圖一次就看得到，不需要因果限制。作者做了一個小實驗：把 ViT 的注意力改成因果模式，ImageNet 的準確率就掉了（原論文 Figure 3b）。\n\n### 影像算長序列嗎？\n\n第一個條件需要一把尺。作者用 Transformer 區塊的計算量來量：輸入有 $L$ 個 token、每個 $D$ 維，MLP 擴張 4 倍時，\n\n$$\n\\text{FLOPs}=24D^2L+4DL^2,\\qquad r_L=\\frac{4DL^2}{24D^2L}=\\frac{L}{6D}\n$$\n\n$r_L$ 大於 1，也就是 $L>6D$ 時，和長度平方相關的那一項就超過線性項，可以算是長序列。\n\n我們拿論文的三個任務算一次，patch 都是 16×16：\n\n```python\nD = {\"ViT-S\": 384, \"ViT-B\": 768}\ntasks = {\"ImageNet 224×224\": (224, 224), \"COCO 800×1280\": (800, 1280), \"ADE20K 512×2048\": (512, 2048)}\nfor name, (h, w) in tasks.items():\n    L = (h // 16) * (w // 16)\n    ratios = \"，\".join(f\"{k} 的 L/6D = {L / (6 * d):.2f}\" for k, d in D.items())\n    print(f\"{name}: {L:,} 個 token；{ratios}\")\n```\n\n```text\nImageNet 224×224: 196 個 token；ViT-S 的 L/6D = 0.09，ViT-B 的 L/6D = 0.04\nCOCO 800×1280: 4,000 個 token；ViT-S 的 L/6D = 1.74，ViT-B 的 L/6D = 0.87\nADE20K 512×2048: 4,096 個 token；ViT-S 的 L/6D = 1.78，ViT-B 的 L/6D = 0.89\n```\n\nImageNet 分類只有 196 個 token，離門檻差了十倍以上。COCO 偵測與 ADE20K 分割約 4K 個 token，對 ViT-S 已經過了門檻，對 ViT-B 也接近了。\n\n於是作者提出兩個假設：\n\n- **假設一**：ImageNet 分類既不長、也不需要因果，用不到 SSM。\n- **假設二**：偵測與分割雖然不需要因果，但序列夠長，SSM 也許還有幫助。\n\n### 把 SSM 拿掉\n\n[![左圖是兩種區塊的結構：Gated CNN 區塊的輸入經過兩個 Linear 分成兩支，一支經過 Conv，另一支經過啟動函數，兩支相乘後再經過 Linear；Mamba 區塊相同，只是 Conv 之後多了啟動函數與 SSM。右圖是 ImageNet Top-1 準確率對計算量的氣泡圖，MambaOut 的曲線位於 VMamba、PlainMamba 與 Vision Mamba 之上](/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png)](/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png)\n\n*原論文 Figure 1，取自 [**v3 PDF 第 1 頁**](https://arxiv.org/pdf/2405.07992v3#page=1)。(a) 省略了正規化與殘差連接；(b) 氣泡大小代表參數量。*\n\n看左圖，兩個區塊幾乎一模一樣。作者把它們寫成同一個式子：\n\n$$\nY=\\big(\\text{TokenMixer}(X'W_1)\\odot\\sigma(X'W_2)\\big)W_3+X\n$$\n\n$X'$ 是正規化後的輸入，一支經過 token mixer，另一支經過啟動函數當作閘門，兩支逐元素相乘，再投影回原來的維度。差別只在 token mixer：\n\n$$\n\\text{TokenMixer}_{\\text{Gated CNN}}(Z)=\\text{Conv}(Z),\\qquad\n\\text{TokenMixer}_{\\text{Mamba}}(Z)=\\text{SSM}\\big(\\sigma(\\text{Conv}(Z))\\big)\n$$\n\nMamba 區塊就是在 Gated CNN 的卷積後面多接一個 SSM。\n\n所以，把 Mamba 區塊的 SSM 拿掉，剩下的就是 Gated CNN。作者用它疊出一系列模型，取名 **MambaOut**：\n\n- token mixer 用 7×7 的 depthwise 卷積，沿用 ConvNeXt；為了實際速度，只對部分通道做卷積，沿用 InceptionNeXt。\n- 像 ResNet 一樣分四個階段，每個階段之間降採樣。\n- 從最小的 Femto 到 Base，共四種大小。\n\n這個設計是一個基準線：不含 SSM，看看能做到多好。要說明的是，它和各個視覺 Mamba 並不是只差一個 SSM，卷積的寫法、區塊設計和各階段配置都不同。所以就算 MambaOut 比較好，嚴格來說證明的是「不用 SSM 也能做到這個水準」；SSM 本身貢獻多少，要在同一個骨幹上拿掉它才量得出來。\n\n## 討論\n\n### ImageNet 分類\n\n訓練照 DeiT 的配方，不用蒸餾，輸入 224×224。挑幾組參數量相近的比較：\n\n| 模型 | 參數（M） | MACs（G） | Top-1（%） |\n| --- | ---: | ---: | ---: |\n| Vim-Ti | 7 | 1.5 | 76.1 |\n| EfficientVMamba-S | 11 | 1.3 | 78.7 |\n| **MambaOut-Femto** | 7 | 1.2 | **78.9** |\n| VMamba-T | 22 | 5.6 | 82.2 |\n| LocalVMamba-T | 26 | 5.7 | 82.7 |\n| **MambaOut-Tiny** | 27 | 4.5 | **82.7** |\n| LocalVMamba-S | 50 | 11.4 | 83.7 |\n| VMambaV9-S | 50 | 8.7 | 83.6 |\n| **MambaOut-Small** | 48 | 9.0 | **84.1** |\n| VMambaV9-B | 89 | 15.4 | 83.9 |\n| **MambaOut-Base** | 85 | 15.8 | **84.2** |\n\n*表：原論文 Table 1 節錄。*\n\n每個規模下，MambaOut 都追平或超過帶 SSM 的視覺 Mamba。以 Small 為例，MambaOut 比 LocalVMamba-S 高 0.4 個百分點，計算量只有後者的 79%；不過和 VMambaV9 相比，MambaOut 的 MACs 略高一些。\n\n原論文 Table 1 裡還有一組值得對照的數字。CAFormer-M36 用 56M 參數、13.2G MACs 拿到 85.2%；參數更多的 VMambaV9-B（89M、15.4G）是 83.9%。也就是說，評估 SSM 值不值得用，最強的卷積與注意力骨幹也得一起放進比較。\n\n\n> [!TIP]\n> \n> CAFormer 與 MetaFormer 這條研究線，也是同一位作者的作品：\n> \n> - **[[21.11] PoolFormer：你需要的是框架！](/blog/papers/vision-transformers/poolformer)**\n> - **[[22.10] CAFormer：MetaFormer 使用說明書](/blog/papers/vision-transformers/caformer)**\n\n\n### 偵測與分割\n\n換成長序列的任務，結果就反過來了。COCO 上用 Mask R-CNN、1× 訓練排程：\n\n| 骨幹 | AP（框） | AP（遮罩） |\n| --- | ---: | ---: |\n| MambaOut-Tiny | 45.1 | 41.0 |\n| VMamba-T | 46.5 | 42.1 |\n| LocalVMamba-T | 46.7 | 42.2 |\n| VMambaV9-T | 47.4 | 42.7 |\n\n*表：原論文 Table 2 節錄。*\n\nMambaOut-Tiny 比 VMamba-T 少了 1.4 個框 AP、1.1 個遮罩 AP。ADE20K 語意分割也類似：MambaOut-Tiny 的單尺度 mIoU 47.4，和 VMamba-T 的 47.3 差不多，但低於 LocalVMamba-T 的 47.9 與 VMambaV9-T 的 48.3。\n\n作者把這解讀成假設二的支持：序列夠長時，SSM 可能有幫上忙。和分類一樣，這裡比較的也是不同的整體架構，這個結果比較適合當成繼續研究的線索。\n\n不過他們也沒有替視覺 Mamba 說太多好話。COCO（Table 2，Mask R-CNN）與 ADE20K（Table 3，UperNet）兩張表上，卷積加注意力的 TransNeXt 都領先同級的視覺 Mamba；例如 ADE20K 單尺度 mIoU，TransNeXt-Tiny 是 51.1。\n\n### 這把尺的限制\n\n$L>6D$ 這個門檻，量的是單一 Transformer 區塊的計算量什麼時候被平方項主導。它是一個方便的判斷，但不能拿來證明某個任務「一定需要」線性時間的模型。本文另外想到一點：偵測與分割的骨幹也常用視窗注意力或階層式降採樣來降低成本，平方項的負擔未必真的出現。\n\n另外，假設二的證據比假設一弱一些。上面 Tiny 組的偵測差距是 1.4 到 2.3 個框 AP、1.1 到 1.7 個遮罩 AP，分割上有些視覺 Mamba 也沒有贏過 MambaOut。這篇對「ImageNet 分類不需要 SSM」的支持很充分；至於「長序列的視覺任務需要 SSM」，比較接近「值得繼續研究」。\n\n## 結論\n\n這篇論文用一個簡單的基準線回答了題目：\n\n1. **作者認為 Mamba 適合長序列、因果式的任務**。固定大小的記憶在短序列上占不到便宜，單向遞迴也只沿著讀的方向累積資訊。\n2. **ImageNet 分類兩個條件都不符合**。不含 SSM 的 MambaOut 在表列各規模都追平或勝過視覺 Mamba；MACs 的高低則依比較對象而異。\n3. **偵測與分割是長序列，SSM 可能有一點幫助**。MambaOut 在這裡輸給最好的視覺 Mamba，但所有視覺 Mamba 又都輸給更強的卷積與注意力混合模型。\n\nVision Mamba 的火苗才正要熊熊燃起，就這樣被掐滅了。\n\n哎，世事無常。\n\n\n> [!TIP]\n> \n> 把 Mamba 搬進視覺的第一批工作之一，可以參考：\n> \n> - **[[24.01] Vim：曼巴的視線](/blog/papers/mamba/vim)**\n","imageSizes":{"/content/papers/assets/0a06070fe70c8178d01ed035d860f87f5caa629d7b13f3eb90b15d154a055398.png":{"width":983,"height":619},"/content/papers/assets/a9e2bb043c6ac8f4558024557394c6a1ec7b9007b52ba19b5d1f7c05b07a3bc7.png":{"width":1112,"height":456}}}