{"notes":[{"id":"classic-cnns/fcdm","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FCDM","subtitle":"只能用 Transformer 嗎？","author":"Z. Yuan","paperDate":"2026-03-10","publishedOn":"2026-04-07","paperTitle":"Reviving ConvNeXt for Efficient Convolutional Diffusion Models","paperUrl":"https://arxiv.org/abs/2603.09408","bibliography":{"title":"Reviving ConvNeXt for Efficient Convolutional Diffusion Models","authors":["Taesung Kwon","Lorenzo Bianchi","Lennart Wittke","Felix Watine","Fabio Carrara","Jong Chul Ye","Romann Weber","Vinicius Azevedo"],"url":"https://arxiv.org/abs/2603.09408","pdfUrl":"https://arxiv.org/pdf/2603.09408v1","codeUrl":"https://github.com/star-kwon/FCDM","preprint":{"id":"2603.09408","submittedOn":"2026-03-10","url":"https://arxiv.org/abs/2603.09408"},"publication":{"venue":"CVPR","year":2026,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2026/html/Kwon_Reviving_ConvNeXt_for_Efficient_Convolutional_Diffusion_Models_CVPR_2026_paper.html"},"citation":"@misc{classiccnnsfcdm2026,\n  title = {{Reviving ConvNeXt for Efficient Convolutional Diffusion Models}},\n  author = {Taesung Kwon and Lorenzo Bianchi and Lennart Wittke and Felix Watine and Fabio Carrara and Jong Chul Ye and Romann Weber and Vinicius Azevedo},\n  year = {2026},\n  eprint = {2603.09408},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2603.09408}\n}\n","citationPath":"/content/papers/citations/classic-cnns/fcdm.7438c4ca3414.bib"},"thumbnail":{"src":"/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png","width":1960,"height":1280},"locale":"zh-hant","path":"/content/papers/zh-hant/classic-cnns/fcdm.5f13c452fb69.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"擴散模型的骨幹在做什麼","text":"擴散模型的骨幹在做什麼","depth":3},{"id":"卷積能不能也這樣放大","text":"卷積能不能也這樣放大","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"fcdm-區塊","text":"FCDM 區塊","depth":3},{"id":"u-形架構只有兩個旋鈕","text":"U 形架構，只有兩個旋鈕","depth":3},{"id":"和-dico-差在哪","text":"和 DiCo 差在哪","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"實驗設定","text":"實驗設定","depth":3},{"id":"每個尺寸都贏-dit","text":"每個尺寸都贏 DiT","depth":3},{"id":"用上-guidance-之後","text":"用上 guidance 之後","depth":3},{"id":"解析度加倍","text":"解析度加倍","depth":3},{"id":"消融","text":"消融","depth":3},{"id":"四張顯卡","text":"四張顯卡","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"classic-cnns/convnext-v2","relation":"referenced","topic":""},{"id":"feature-fusion/unet","relation":"referenced","topic":""}]},{"id":"classic-cnns/convnext","category":"classic-cnns","legacyPath":"classic-cnns/2201-convnext","legacySlug":"2201-convnext","legacyPaths":[],"title":"ConvNeXt","subtitle":"讓卷積再次偉大","author":"Z. Yuan","paperDate":"2022-01-10","publishedOn":"2024-06-30","paperTitle":"A ConvNet for the 2020s","paperUrl":"https://arxiv.org/abs/2201.03545","bibliography":{"title":"A ConvNet for the 2020s","authors":["Zhuang Liu","Hanzi Mao","Chao-Yuan Wu","Christoph Feichtenhofer","Trevor Darrell","Saining Xie"],"url":"https://arxiv.org/abs/2201.03545","pdfUrl":"https://arxiv.org/pdf/2201.03545","codeUrl":"https://github.com/facebookresearch/ConvNeXt","preprint":{"id":"2201.03545","submittedOn":"2022-01-10","url":"https://arxiv.org/abs/2201.03545"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://arxiv.org/abs/2201.03545"},"citation":"@misc{classiccnnsconvnext2022,\n  title = {{A ConvNet for the 2020s}},\n  author = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie},\n  year = {2022},\n  eprint = {2201.03545},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2201.03545}\n}\n","citationPath":"/content/papers/citations/classic-cnns/convnext.801b4060b807.bib"},"thumbnail":{"src":"/content/papers/assets/37882616c5c72df33699003256d14bf7c58c8ea5d1e9e4fb5b1c33268c9a6923.jpg","width":1224,"height":432},"locale":"zh-hant","path":"/content/papers/zh-hant/classic-cnns/convnext.667fd92d121a.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"重新設計網路架構","text":"重新設計網路架構","depth":3},{"id":"引入現代訓練技術","text":"引入現代訓練技術","depth":3},{"id":"宏觀設計","text":"宏觀設計","depth":3},{"id":"resnext-ify","text":"ResNeXt-ify","depth":3},{"id":"inverted-bottleneck","text":"Inverted Bottleneck","depth":3},{"id":"大核卷積","text":"大核卷積","depth":3},{"id":"微觀設計","text":"微觀設計","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"總結模型配置","text":"總結模型配置","depth":3},{"id":"imagenet-上的性能","text":"ImageNet 上的性能","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/convmixer","relation":"comparison","topic":"用卷積混合影像切塊"},{"id":"vision-transformers/caformer","relation":"comparison","topic":"用卷積混合影像切塊"},{"id":"reparameterization/replknet","relation":"comparison","topic":"大卷積核與簡化骨架"}]},{"id":"classic-cnns/convnext-v2","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"ConvNeXt V2","subtitle":"卷積也會填空","author":"Z. Yuan","paperDate":"2023-01-02","publishedOn":"2025-09-27","world":"external/convnext-v2","paperTitle":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders","paperUrl":"https://arxiv.org/abs/2301.00808","bibliography":{"title":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders","authors":["Sanghyun Woo","Shoubhik Debnath","Ronghang Hu","Xinlei Chen","Zhuang Liu","In So Kweon","Saining Xie"],"url":"https://arxiv.org/abs/2301.00808","pdfUrl":"https://arxiv.org/pdf/2301.00808v1","codeUrl":null,"preprint":{"id":"2301.00808","submittedOn":"2023-01-02","url":"https://arxiv.org/abs/2301.00808"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Woo_ConvNeXt_V2_Co-Designing_and_Scaling_ConvNets_With_Masked_Autoencoders_CVPR_2023_paper.html"},"citation":"@misc{classiccnnsconvnextv22023,\n  title = {{ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders}},\n  author = {Sanghyun Woo and Shoubhik Debnath and Ronghang Hu and Xinlei Chen and Zhuang Liu and In So Kweon and Saining Xie},\n  year = {2023},\n  eprint = {2301.00808},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2301.00808}\n}\n","citationPath":"/content/papers/citations/classic-cnns/convnext-v2.065fc6e2e545.bib"},"thumbnail":{"src":"/content/papers/assets/0af11b7bcef637ee0205fe47b31901b1b775a0336ccab0042e73e8d9643589c0.png","width":1640,"height":600},"locale":"zh-hant","path":"/content/papers/zh-hant/classic-cnns/convnext-v2.dbf68ab931f9.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"mae-為什麼不好套","text":"MAE 為什麼不好套","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"fcmae全卷積的-mae","text":"FCMAE：全卷積的 MAE","depth":3},{"id":"只換預訓練不夠","text":"只換預訓練，不夠","depth":3},{"id":"特徵崩塌","text":"特徵崩塌","depth":3},{"id":"grn讓通道互相競爭","text":"GRN：讓通道互相競爭","depth":3},{"id":"convnext-v2-區塊","text":"ConvNeXt V2 區塊","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"架構和預訓練要一起改","text":"架構和預訓練要一起改","depth":3},{"id":"和-transformer-的-mae-比","text":"和 Transformer 的 MAE 比","depth":3},{"id":"偵測與分割","text":"偵測與分割","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"vision-transformers/mae","relation":"referenced","topic":""}]},{"id":"feature-fusion/unet","category":"feature-fusion","legacyPath":"feature-fusion/1505-unet","legacySlug":"1505-unet","legacyPaths":[],"title":"U-Net","subtitle":"融合的序幕","author":"Z. Yuan","paperDate":"2015-05-18","publishedOn":"2024-08-15","paperTitle":"U-Net: Convolutional Networks for Biomedical Image Segmentation","paperUrl":"https://arxiv.org/abs/1505.04597","bibliography":{"title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","authors":["Olaf Ronneberger","Philipp Fischer","Thomas Brox"],"url":"https://arxiv.org/abs/1505.04597","pdfUrl":"https://arxiv.org/pdf/1505.04597","codeUrl":null,"preprint":{"id":"1505.04597","submittedOn":"2015-05-18","url":"https://arxiv.org/abs/1505.04597"},"publication":{"venue":"MICCAI","year":2015,"kind":"conference","url":"https://doi.org/10.1007/978-3-319-24574-4_28"},"citation":"@misc{featurefusionunet2015,\n  title = {{U-Net: Convolutional Networks for Biomedical Image Segmentation}},\n  author = {Olaf Ronneberger and Philipp Fischer and Thomas Brox},\n  year = {2015},\n  eprint = {1505.04597},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1505.04597}\n}\n","citationPath":"/content/papers/citations/feature-fusion/unet.b015405a1c46.bib"},"thumbnail":{"src":"/content/papers/assets/0b302bf8ed079d85b3a07562463380875eb3d9dfc631684549000c38d16a5f38.jpg","width":1224,"height":824},"locale":"zh-hant","path":"/content/papers/zh-hant/feature-fusion/unet.89cc26307d56.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"討論","text":"討論","depth":2},{"id":"isbi-cell-tracking-challenge-2015","text":"ISBI Cell Tracking Challenge 2015","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"segmentation/fcn","relation":"background","topic":"密集預測與編解碼器的跳接"},{"id":"feature-fusion/unetpp","relation":"next","topic":"密集預測與編解碼器的跳接"},{"id":"feature-fusion/trans-unet","relation":"comparison","topic":"編解碼器如何跨尺度傳遞特徵"}]}],"text":"擴散模型的骨幹，這幾年走了一條很清楚的路。\n\n最早的 DDPM、ADM，還有 Stable Diffusion 背後的潛在擴散模型（LDM），用的都是卷積加注意力的 U-Net。後來的 DiT 把卷積全部拿掉，換成一疊 Transformer 區塊，結果放大得很漂亮。後來的 SD3、FLUX 這些文生圖模型，骨幹也都是 Transformer。\n\n可是分類那邊，ConvNeXt 早就證明了卷積網路調得好，可以和 Transformer 打平。那擴散模型為什麼不能用純卷積？\n\n這篇來自 KAIST、蘇黎世聯邦理工學院（ETH Zürich）和義大利的研究者。他們直接把 ConvNeXt 改成擴散模型的骨幹，用 DiT 的訓練和評估流程，從小模型一路比到大模型。\n\n\n> [!TIP]\n> \n> ConvNeXt 的設計、ConvNeXt V2 的 GRN，以及 U-Net 的結構，可以先看這幾篇：\n> \n> - **[[22.01] ConvNeXt：讓卷積再次偉大](/blog/papers/classic-cnns/convnext)**\n> - **[[23.01] ConvNeXt V2：卷積也會填空](/blog/papers/classic-cnns/convnext-v2)**\n> - **[[15.05] U-Net：融合的序幕](/blog/papers/feature-fusion/unet)**\n\n\n## 定義問題\n\n### 擴散模型的骨幹在做什麼\n\n先交代一下背景。擴散模型從一張純雜訊開始，一步步去雜訊，最後得到一張圖。每一步都要呼叫一次骨幹網路，輸入是目前帶雜訊的圖、時間步 $t$ 和條件（這篇用的是 ImageNet 類別 $y$），輸出是預測的雜訊。\n\n為了省計算，現在多半在潛在空間裡做。256×256 的圖先用 VAE 壓成 32×32×4 的潛在表示，擴散和去雜訊都在這個小張量上進行，最後再解碼回影像。\n\nDiT 把潛在表示切成小塊當 token，送進 Transformer。它被廣泛採用，是因為只要調區塊數、通道數、頭數和切塊大小幾個參數就能放大，時間和類別的條件也用 adaLN-Zero 乾淨地注入每個區塊。缺點是貴，DiT-XL/2 每次前向傳播要 118.6 GFLOPs，DiT 最好的那組成績訓練了 700 萬步。\n\n### 卷積能不能也這樣放大\n\n之前已經有人回頭試卷積骨幹，例如 DiC 用 3×3 卷積的 U-Net，DiCo 用 3×3 可分離卷積加上一種通道注意力，都比 DiT 省算力。這篇想問的是更直接的版本。ConvNeXt 那套在分類上驗證過的設計，原封不動地搬過來，能不能像 DiT 一樣好放大，又比 DiT 省？\n\n## 解決問題\n\n### FCDM 區塊\n\n[![本文整理的示意圖。左邊是 FCDM 區塊，由上到下是 7×7 depthwise 卷積、不帶仿射參數的 LayerNorm、藍色的「乘上 (1 + scale) 再加 shift」、把通道從 C 放大到 rC 的 1×1 卷積、GELU、GRN、把通道縮回 C 的 1×1 卷積、藍色的「乘上 gate」，最後和輸入相加。右側一條藍色的條件分支：類別 y 和時間 t 的嵌入 c，經過 SiLU 和線性層，產生 shift、scale 和 gate，線性層標註初始化為零；下方註明初始化為零時 gate 等於 0，整個區塊一開始就是恆等函數。右邊是 U 形架構：輸入潛在表示經過 L 個區塊、C 個通道，降採樣兩倍後是 2L 個區塊、2C 個通道，再降一次是中間的 4L 個區塊、4C 個通道，然後對稱地升回去，同層之間用虛線的跳接相連。](/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png)](/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png)\n\n*圖：本文整理。依原論文第 3.1 節與官方程式碼繪製。*\n\n區塊幾乎就是 ConvNeXt V2 的區塊。7×7 的 depthwise 卷積，接 LayerNorm，再用兩個 1×1 卷積先放大、再縮回，中間夾著 GELU 和 GRN，最後加上殘差。官方程式碼的放大倍數 $r$ 是 3。\n\nConvNeXt 本來是做分類的，沒有地方放時間和類別的條件。作者學 DiT 的 adaLN-Zero，讓類別和時間的嵌入經過一個小的線性層，產生 shift、scale、gate 三組參數。LayerNorm 拿掉自己的仿射參數，改用 $h\\cdot(1+\\text{scale})+\\text{shift}$；區塊輸出再乘上 gate，然後才和殘差相加。這個線性層初始化為零，所以剛開始訓練時 gate 是 0，每個區塊都是恆等函數，網路再深也能穩定起步。\n\n我們照這個描述寫一個區塊，看看初始化和條件的效果：\n\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass GRN(nn.Module):                                  # ConvNeXt V2 的全域響應正規化\n    def __init__(self, c):\n        super().__init__()\n        self.gamma = nn.Parameter(torch.zeros(1, c, 1, 1))\n        self.beta = nn.Parameter(torch.zeros(1, c, 1, 1))\n\n    def forward(self, x):\n        g = x.norm(p=2, dim=(2, 3), keepdim=True)\n        return self.gamma * (x * g / (g.mean(1, keepdim=True) + 1e-6)) + self.beta + x\n\nclass FCDMBlock(nn.Module):\n    def __init__(self, c, r=3):\n        super().__init__()\n        self.dw = nn.Conv2d(c, c, 7, padding=3, groups=c)\n        self.pw1, self.pw2 = nn.Conv2d(c, r * c, 1), nn.Conv2d(r * c, c, 1)\n        self.grn = GRN(r * c)\n        self.ada = nn.Sequential(nn.SiLU(), nn.Linear(c, 3 * c))\n        nn.init.zeros_(self.ada[1].weight)              # 條件分支從 0 開始\n        nn.init.zeros_(self.ada[1].bias)\n\n    def forward(self, x, cond):\n        shift, scale, gate = self.ada(cond)[:, :, None, None].chunk(3, dim=1)\n        h = F.layer_norm(self.dw(x).permute(0, 2, 3, 1), (x.shape[1],)).permute(0, 3, 1, 2)\n        h = h * (1 + scale) + shift                     # 條件決定每個通道怎麼縮放、平移\n        h = self.pw2(self.grn(F.gelu(self.pw1(h))))\n        return x + gate * h                             # gate 也由條件決定\n\ntorch.manual_seed(0)\nc = 32\nblock = FCDMBlock(c)\nx = torch.randn(2, c, 16, 16)\ncond_a, cond_b = torch.randn(1, c).expand(2, c), torch.randn(1, c).expand(2, c)   # 兩組不同的類別＋時間嵌入\n\nwith torch.no_grad():\n    print(\"剛初始化：輸出和輸入的最大差距\", f\"{(block(x, cond_a) - x).abs().max():.1e}\")\n    nn.init.normal_(block.ada[1].weight, std=0.1)    # 模擬訓練一陣子之後\n    ya, yb = block(x, cond_a), block(x, cond_b)\n    print(\"訓練後：同一張輸入、不同條件的輸出差距\", f\"{(ya - yb).abs().max():.2f}\")\n```\n\n```text\n剛初始化：輸出和輸入的最大差距 0.0e+00\n訓練後：同一張輸入、不同條件的輸出差距 0.85\n```\n\n剛初始化時，區塊完全不改變輸入。條件分支的權重長出來之後，同一張輸入在不同的類別和時間下，會被處理成不同的樣子。條件就是從這三組參數注入的。\n\n### U 形架構，只有兩個旋鈕\n\n區塊排成 U 形。往下走三層，通道數是 $C$、$2C$、$4C$，區塊數是 $L$、$2L$、$4L$，每降一次採樣就兩者都加倍；再對稱地升回來，同層用跳接相連。整個架構只有 $L$ 和 $C$ 兩個超參數。\n\n為了和 DiT 公平比較，四個尺寸的參數量都對齊 DiT：\n\n| 模型 | $L$ | $C$ | 參數量 | FLOPs | 相對 DiT 的 FLOPs |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| FCDM-S | 2 | 128 | 32.7M | 3.1G | 50.8% |\n| FCDM-B | 2 | 256 | 127.7M | 12.2G | 53.0% |\n| FCDM-L | 2 | 512 | 504.5M | 48.3G | 59.9% |\n| FCDM-XL | 3 | 512 | 698.8M | 64.6G | 54.5% |\n\n*表：摘自原論文 Table 2，FLOPs 在 256×256 解析度下計算。*\n\n參數量一樣，FLOPs 大約只有 DiT 的一半。\n\n### 和 DiCo 差在哪\n\nDiCo 是之前最好的卷積擴散模型，結構和 FCDM 很像，作者花了一節說明差在哪裡。\n\nDiCo 的卷積模組從頭到尾維持原本的通道數。FCDM 用 ConvNeXt 的倒置瓶頸，先在原本的通道數上做 depthwise 卷積，再放大通道，所以 depthwise 卷積的成本不受放大倍數影響。DiCo 用一個額外的 1×1 卷積做通道注意力，讓各通道的響應更多樣；FCDM 用 GRN 達到類似的效果，幾乎不用額外參數。DiCo 還有一個獨立的前饋模組，FCDM 的倒置瓶頸已經做了這件事，就省掉了。\n\n結果是同樣參數量下，FCDM 的 FLOPs 大約是 DiCo 的四分之三。\n\n## 討論\n\n### 實驗設定\n\n訓練完全照 DiT 的流程。AdamW，學習率固定 $10^{-4}$，沒有權重衰減，批次 256，資料增強只有水平翻轉，權重取 EMA。評估用 FID，從 5 萬張生成圖和真實圖的特徵分布算距離，越低越好。\n\n### 每個尺寸都贏 DiT\n\n先看訓練 40 萬步、不用 classifier-free guidance 的結果：\n\n| 模型 | FLOPs | 推論吞吐量 | FID |\n| --- | ---: | ---: | ---: |\n| DiT-S/2 | 6.1G | 1234 | 68.4 |\n| FCDM-S | 3.1G | 2687 | **48.5** |\n| DiT-B/2 | 23.0G | 380 | 43.5 |\n| FCDM-B | 12.2G | 1002 | **26.2** |\n| DiT-L/2 | 80.7G | 115 | 23.3 |\n| FCDM-L | 48.3G | 381 | **13.8** |\n| DiT-XL/2 | 118.6G | 81 | 19.5 |\n| FCDM-XL | 64.6G | 273 | **10.7** |\n| DiT-XL/2（700 萬步） | 118.6G | 81 | 9.6 |\n| FCDM-XL（100 萬步） | 64.6G | 273 | **7.9** |\n\n*表：摘自原論文 Table 1 與 Table 3，吞吐量的單位是論文報告的 it/s。*\n\n同樣的步數，FCDM 每個尺寸的 FID 都比 DiT 低得多，推論吞吐量是 DiT 的 2.2 到 3.4 倍。最後兩列最能說明收斂速度，FCDM-XL 訓練 100 萬步，FID 就比訓練 700 萬步的 DiT-XL/2 還低。\n\n和其他卷積骨幹比，差距就小多了。同樣 40 萬步，DiCo-L 的 FID 是 13.66，比 FCDM-L 的 13.83 還好一點；XL 這一級 FCDM 是 10.72，DiCo-XL 是 11.67，FCDM 的 FLOPs 少了四分之一。小模型的吞吐量則是 DiC 最高，它用的是硬體最佳化得最好的標準 3×3 卷積。\n\n### 用上 guidance 之後\n\n把 FCDM-XL 訓練到 200 萬步（400 個 epoch），加上 classifier-free guidance，和各家已發表的結果比：\n\n| 模型 | 訓練 epoch | FLOPs | FID | IS |\n| --- | ---: | ---: | ---: | ---: |\n| DiT-XL/2 | 1400 | 118.6G | 2.27 | 278.2 |\n| SiT-XL/2 | 1400 | 118.6G | 2.06 | 277.5 |\n| DiCo-XL | 750 | 87.3G | 2.05 | 282.2 |\n| FCDM-XL | 400 | 64.6G | **2.03** | **285.7** |\n\n*表：摘自原論文 Table 4 的部分列，ImageNet 256×256。*\n\nFID 2.03 比 DiT-XL/2 的 2.27 好，訓練 epoch 不到它的三分之一。不過和 SiT、DiCo 比，差距只在小數點後第二位。作者自己也寫明，FCDM 在 DiT 這套實驗設定下勝過同級對手，但還沒超過 EDM-2、Simpler Diffusion 這些最新的頂尖結果。\n\n### 解析度加倍\n\n在 512×512 上，FCDM-XL 訓練 100 萬步的 FID 是 7.46，DiT-XL/2 訓練 300 萬步是 12.03，DiCo-XL 訓練 130 萬步是 8.10。\n\n更有意思的是速度的變化。解析度從 256 升到 512，DiT-XL/2 的吞吐量從 80.5 掉到 18.6，少了四倍多；FCDM-XL 從 272.7 掉到 129.6，只少了一半左右。注意力的計算量隨 token 數平方成長，卷積只隨像素數線性成長，解析度越高，差距越大。\n\n### 消融\n\n用 FCDM-L 訓練 20 萬步，每次改一個地方，看 FID：\n\n| 改動 | FID |\n| --- | ---: |\n| FCDM-L（7×7 depthwise、GRN、倒置瓶頸） | **19.97** |\n| depthwise 卷積改成 5×5 | 20.48 |\n| depthwise 卷積改成 3×3 | 21.28 |\n| GRN 換成 DiCo 的通道注意力 | 23.85 |\n| 加上 DiCo 的前饋模組 | 28.52 |\n| 拿掉倒置瓶頸 | 28.76 |\n| 換成 ResNet 區塊 | 31.14 |\n\n*表：摘自原論文 Table 6。後四列調整了通道數，讓 FLOPs 一致。*\n\n卷積核越小越差，大卷積核在生成任務上一樣有用。把 DiCo 的設計搬進來都會變差，拿掉倒置瓶頸或換回 ResNet 區塊更是差了一大截。ConvNeXt 這套設計，在生成任務上幾乎每一項都還站得住。\n\n### 四張顯卡\n\n論文特別強調訓練成本。FCDM-XL 在四張 RTX 4090 上，開梯度檢查點、批次 256，每秒大約跑 0.9 步；批次 256 也放得進一張 40GB 的 A100。\n\n照這個速度推算，40 萬步大約 5 天，200 萬步大約 26 天。這是本文用論文給的數字算的，不是論文的報告。對一般實驗室來說，這是摸得到的規模。\n\n官方 repo 提供了訓練和取樣的程式碼，授權是 CC BY-NC，目前沒有附上預訓練權重。\n\n## 結論\n\nFCDM 幾乎沒有發明新東西。ConvNeXt V2 的區塊、DiT 的 adaLN-Zero 條件注入、一個只有兩個旋鈕的 U 形架構，三樣都是現成的。組起來之後，FLOPs 只有 DiT 的一半，收斂快了好幾倍，解析度越高優勢越明顯。\n\n比 2.03 更有說服力的是消融表。把 DiCo 的改良搬進來會變差，把 ConvNeXt 的設計拿掉也會變差，一套為分類設計的區塊，在生成任務上幾乎原封不動地成立。\n\n所以標題那個問題的答案，至少在 ImageNet 的類別條件生成上，是「不一定」。\n\n文生圖就還早。論文附錄只在 MS-COCO 上訓練了 10 萬步、看了幾張生成圖，條件也只用了池化過的 CLIP 文字向量；要像 SD3 那樣接上完整的文字序列，作者自己也列為之後的工作。\n","imageSizes":{"/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png":{"width":1960,"height":1280}}}