{"notes":[{"id":"classic-cnns/fcdm","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FCDM","subtitle":"Must It Be a Transformer?","author":"Z. Yuan","paperDate":"2026-03-10","publishedOn":"2026-04-07","paperTitle":"Reviving ConvNeXt for Efficient Convolutional Diffusion Models","paperUrl":"https://arxiv.org/abs/2603.09408","bibliography":{"title":"Reviving ConvNeXt for Efficient Convolutional Diffusion Models","authors":["Taesung Kwon","Lorenzo Bianchi","Lennart Wittke","Felix Watine","Fabio Carrara","Jong Chul Ye","Romann Weber","Vinicius Azevedo"],"url":"https://arxiv.org/abs/2603.09408","pdfUrl":"https://arxiv.org/pdf/2603.09408v1","codeUrl":"https://github.com/star-kwon/FCDM","preprint":{"id":"2603.09408","submittedOn":"2026-03-10","url":"https://arxiv.org/abs/2603.09408"},"publication":{"venue":"CVPR","year":2026,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2026/html/Kwon_Reviving_ConvNeXt_for_Efficient_Convolutional_Diffusion_Models_CVPR_2026_paper.html"},"citation":"@misc{classiccnnsfcdm2026,\n  title = {{Reviving ConvNeXt for Efficient Convolutional Diffusion Models}},\n  author = {Taesung Kwon and Lorenzo Bianchi and Lennart Wittke and Felix Watine and Fabio Carrara and Jong Chul Ye and Romann Weber and Vinicius Azevedo},\n  year = {2026},\n  eprint = {2603.09408},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2603.09408}\n}\n","citationPath":"/content/papers/citations/classic-cnns/fcdm.7438c4ca3414.bib"},"thumbnail":{"src":"/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png","width":1960,"height":1280},"locale":"en","path":"/content/papers/en/classic-cnns/fcdm.d74834ecfcd3.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"what-a-diffusion-backbone-does","text":"What a Diffusion Backbone Does","depth":3},{"id":"can-convolution-scale-this-way-too","text":"Can Convolution Scale This Way Too?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"the-fcdm-block","text":"The FCDM Block","depth":3},{"id":"a-u-shape-with-only-two-knobs","text":"A U Shape with Only Two Knobs","depth":3},{"id":"how-it-differs-from-dico","text":"How It Differs from DiCo","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"experimental-setup","text":"Experimental Setup","depth":3},{"id":"beating-dit-at-every-size","text":"Beating DiT at Every Size","depth":3},{"id":"with-guidance","text":"With Guidance","depth":3},{"id":"double-the-resolution","text":"Double the Resolution","depth":3},{"id":"ablation","text":"Ablation","depth":3},{"id":"four-graphics-cards","text":"Four Graphics Cards","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"classic-cnns/convnext-v2","relation":"referenced","topic":""},{"id":"feature-fusion/unet","relation":"referenced","topic":""}]},{"id":"classic-cnns/convnext","category":"classic-cnns","legacyPath":"classic-cnns/2201-convnext","legacySlug":"2201-convnext","legacyPaths":[],"title":"ConvNeXt","subtitle":"Making Convolutions Great Again","author":"Z. Yuan","paperDate":"2022-01-10","publishedOn":"2024-06-30","paperTitle":"A ConvNet for the 2020s","paperUrl":"https://arxiv.org/abs/2201.03545","bibliography":{"title":"A ConvNet for the 2020s","authors":["Zhuang Liu","Hanzi Mao","Chao-Yuan Wu","Christoph Feichtenhofer","Trevor Darrell","Saining Xie"],"url":"https://arxiv.org/abs/2201.03545","pdfUrl":"https://arxiv.org/pdf/2201.03545","codeUrl":"https://github.com/facebookresearch/ConvNeXt","preprint":{"id":"2201.03545","submittedOn":"2022-01-10","url":"https://arxiv.org/abs/2201.03545"},"publication":{"venue":"CVPR","year":2022,"kind":"conference","url":"https://arxiv.org/abs/2201.03545"},"citation":"@misc{classiccnnsconvnext2022,\n  title = {{A ConvNet for the 2020s}},\n  author = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie},\n  year = {2022},\n  eprint = {2201.03545},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2201.03545}\n}\n","citationPath":"/content/papers/citations/classic-cnns/convnext.801b4060b807.bib"},"thumbnail":{"src":"/content/papers/assets/37882616c5c72df33699003256d14bf7c58c8ea5d1e9e4fb5b1c33268c9a6923.jpg","width":1224,"height":432},"locale":"en","path":"/content/papers/en/classic-cnns/convnext.1a8312c7b164.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"redesigning-network-architecture","text":"Redesigning Network Architecture","depth":3},{"id":"introducing-modern-training-techniques","text":"Introducing Modern Training Techniques","depth":3},{"id":"macro-design","text":"Macro Design","depth":3},{"id":"resnext-ify","text":"ResNeXt-ify","depth":3},{"id":"inverted-bottleneck","text":"Inverted Bottleneck","depth":3},{"id":"large-kernel-convolution","text":"Large Kernel Convolution","depth":3},{"id":"micro-design","text":"Micro Design","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"summary-of-model-configurations","text":"Summary of Model Configurations","depth":3},{"id":"performance-on-imagenet","text":"Performance on ImageNet","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"vision-transformers/convmixer","relation":"comparison","topic":"Convolution for mixing visual patches"},{"id":"vision-transformers/caformer","relation":"comparison","topic":"Convolution for mixing visual patches"},{"id":"reparameterization/replknet","relation":"comparison","topic":"Large kernels and simpler convolutional backbones"}]},{"id":"classic-cnns/convnext-v2","category":"classic-cnns","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"ConvNeXt V2","subtitle":"Convolutions Can Fill in the Blanks Too","author":"Z. Yuan","paperDate":"2023-01-02","publishedOn":"2025-09-27","world":"external/convnext-v2","paperTitle":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders","paperUrl":"https://arxiv.org/abs/2301.00808","bibliography":{"title":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders","authors":["Sanghyun Woo","Shoubhik Debnath","Ronghang Hu","Xinlei Chen","Zhuang Liu","In So Kweon","Saining Xie"],"url":"https://arxiv.org/abs/2301.00808","pdfUrl":"https://arxiv.org/pdf/2301.00808v1","codeUrl":null,"preprint":{"id":"2301.00808","submittedOn":"2023-01-02","url":"https://arxiv.org/abs/2301.00808"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Woo_ConvNeXt_V2_Co-Designing_and_Scaling_ConvNets_With_Masked_Autoencoders_CVPR_2023_paper.html"},"citation":"@misc{classiccnnsconvnextv22023,\n  title = {{ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders}},\n  author = {Sanghyun Woo and Shoubhik Debnath and Ronghang Hu and Xinlei Chen and Zhuang Liu and In So Kweon and Saining Xie},\n  year = {2023},\n  eprint = {2301.00808},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2301.00808}\n}\n","citationPath":"/content/papers/citations/classic-cnns/convnext-v2.065fc6e2e545.bib"},"thumbnail":{"src":"/content/papers/assets/0af11b7bcef637ee0205fe47b31901b1b775a0336ccab0042e73e8d9643589c0.png","width":1640,"height":600},"locale":"en","path":"/content/papers/en/classic-cnns/convnext-v2.0556702ca43f.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"why-mae-doesnt-carry-over-easily","text":"Why MAE Doesn't Carry Over Easily","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"fcmae-a-fully-convolutional-mae","text":"FCMAE: A Fully Convolutional MAE","depth":3},{"id":"changing-only-the-pretraining-isnt-enough","text":"Changing Only the Pretraining Isn't Enough","depth":3},{"id":"feature-collapse","text":"Feature Collapse","depth":3},{"id":"grn-making-channels-compete","text":"GRN: Making Channels Compete","depth":3},{"id":"the-convnext-v2-block","text":"The ConvNeXt V2 Block","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"architecture-and-pretraining-have-to-change-together","text":"Architecture and Pretraining Have to Change Together","depth":3},{"id":"against-transformers-with-mae","text":"Against Transformers with MAE","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"classic-cnns/convnext","relation":"referenced","topic":""},{"id":"vision-transformers/mae","relation":"referenced","topic":""}]},{"id":"feature-fusion/unet","category":"feature-fusion","legacyPath":"feature-fusion/1505-unet","legacySlug":"1505-unet","legacyPaths":[],"title":"U-Net","subtitle":"The Dawn of Integration","author":"Z. Yuan","paperDate":"2015-05-18","publishedOn":"2024-08-15","paperTitle":"U-Net: Convolutional Networks for Biomedical Image Segmentation","paperUrl":"https://arxiv.org/abs/1505.04597","bibliography":{"title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","authors":["Olaf Ronneberger","Philipp Fischer","Thomas Brox"],"url":"https://arxiv.org/abs/1505.04597","pdfUrl":"https://arxiv.org/pdf/1505.04597","codeUrl":null,"preprint":{"id":"1505.04597","submittedOn":"2015-05-18","url":"https://arxiv.org/abs/1505.04597"},"publication":{"venue":"MICCAI","year":2015,"kind":"conference","url":"https://doi.org/10.1007/978-3-319-24574-4_28"},"citation":"@misc{featurefusionunet2015,\n  title = {{U-Net: Convolutional Networks for Biomedical Image Segmentation}},\n  author = {Olaf Ronneberger and Philipp Fischer and Thomas Brox},\n  year = {2015},\n  eprint = {1505.04597},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1505.04597}\n}\n","citationPath":"/content/papers/citations/feature-fusion/unet.b015405a1c46.bib"},"thumbnail":{"src":"/content/papers/assets/0b302bf8ed079d85b3a07562463380875eb3d9dfc631684549000c38d16a5f38.jpg","width":1224,"height":824},"locale":"en","path":"/content/papers/en/feature-fusion/unet.cd213004394b.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"discussion","text":"Discussion","depth":2},{"id":"isbi-cell-tracking-challenge-2015","text":"ISBI Cell Tracking Challenge 2015","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"segmentation/fcn","relation":"background","topic":"Dense prediction and encoder-decoder skip connections"},{"id":"feature-fusion/unetpp","relation":"next","topic":"Dense prediction and encoder-decoder skip connections"},{"id":"feature-fusion/trans-unet","relation":"comparison","topic":"Encoder-decoder features across scales"}]}],"text":"Diffusion backbones have followed a clear path in recent years.\n\nEarly DDPM and ADM models, and the latent diffusion model (LDM) behind Stable Diffusion, all use convolution-and-attention U-Nets. DiT later removes convolution entirely in favor of stacked Transformer blocks, scaling impressively. Later text-to-image models such as SD3 and FLUX also use Transformer backbones.\n\nIn classification, though, ConvNeXt already showed that a well-tuned convolutional network can match Transformers. Why couldn't diffusion use pure convolution too?\n\nThis paper comes from researchers at KAIST, ETH Zürich, and in Italy. They directly adapt ConvNeXt into a diffusion backbone, using DiT's training and evaluation pipeline to compare from small through large models.\n\n\n> [!TIP]\n> \n> For ConvNeXt's design, ConvNeXt V2's GRN, and the U-Net structure, see:\n> \n> - **[[22.01] ConvNeXt: Making Convolutions Great Again](/blog/papers/classic-cnns/convnext)**\n> - **[[23.01] ConvNeXt V2: Convolutions Can Fill in the Blanks Too](/blog/papers/classic-cnns/convnext-v2)**\n> - **[[15.05] U-Net: The Dawn of Integration](/blog/papers/feature-fusion/unet)**\n\n\n## Defining the Problem\n\n### What a Diffusion Backbone Does\n\nFirst, some background. A diffusion model starts with pure noise and denoises step by step to produce an image. Every step calls the backbone once, taking the current noisy image, timestep $t$, and condition—ImageNet class $y$ in this paper—and predicting the noise.\n\nTo save compute, this usually happens in latent space. A VAE first compresses a 256×256 image into a 32×32×4 latent representation. Diffusion and denoising operate on this small tensor, then the result is decoded into an image.\n\nDiT splits the latents into patch tokens and feeds them into a Transformer. It is widely used because scaling takes just a few knobs: block count, channel count, head count, and patch size. adaLN-Zero also cleanly injects time and class conditions into every block. The cost is high: DiT-XL/2 takes 118.6 GFLOPs per forward pass, and DiT's best result uses 7 million training steps.\n\n### Can Convolution Scale This Way Too?\n\nOthers have revisited convolutional backbones. DiC uses a U-Net with 3×3 convolutions; DiCo uses 3×3 separable convolutions and a form of channel attention. Both use less compute than DiT. This paper asks a more direct question: can the ConvNeXt design already validated in classification transfer largely unchanged, scale as easily as DiT, and cost less?\n\n## Solving the Problem\n\n### The FCDM Block\n\n[![Diagram compiled for this note. On the left, an FCDM block runs through 7×7 depthwise convolution, affine-free LayerNorm, blue modulation “multiply by (1 + scale), add shift,” a 1×1 convolution expanding C to rC, GELU, GRN, a 1×1 convolution returning to C, a blue gate multiplication, and residual addition. A blue condition branch takes embedding c of class y and time t through SiLU and a zero-initialized linear layer to produce shift, scale, and gate. A note says that a zero-initialized gate makes the block an identity at initialization. On the right is a U-shaped architecture: latent input passes through L blocks at C channels, downsamples to 2L blocks and 2C channels, then to a middle level of 4L blocks and 4C channels, before ascending symmetrically, with dashed skip connections between matching levels.](/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png)](/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png)\n\n*Figure: compiled for this note from Section 3.1 of the paper and the official code.*\n\nThe block is almost ConvNeXt V2: 7×7 depthwise convolution, LayerNorm, then two 1×1 convolutions expanding and compressing channels, with GELU and GRN between them, followed by a residual connection. The official code uses an expansion ratio $r$ of 3.\n\nConvNeXt was built for classification, with no place for time and class conditions. Following DiT's adaLN-Zero, the authors pass class and time embeddings through a small linear layer to produce shift, scale, and gate parameters. LayerNorm drops its own affine parameters and instead uses $h\\cdot(1+\\text{scale})+\\text{shift}$. The block output is multiplied by the gate before residual addition. The linear layer is zero-initialized, so the initial gate is 0 and each block starts as the identity, giving stable initialization even at great depth.\n\nImplement a block from this description and check initialization and conditioning:\n\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nclass GRN(nn.Module):                                  # ConvNeXt V2's Global Response Normalization\n    def __init__(self, c):\n        super().__init__()\n        self.gamma = nn.Parameter(torch.zeros(1, c, 1, 1))\n        self.beta = nn.Parameter(torch.zeros(1, c, 1, 1))\n\n    def forward(self, x):\n        g = x.norm(p=2, dim=(2, 3), keepdim=True)\n        return self.gamma * (x * g / (g.mean(1, keepdim=True) + 1e-6)) + self.beta + x\n\nclass FCDMBlock(nn.Module):\n    def __init__(self, c, r=3):\n        super().__init__()\n        self.dw = nn.Conv2d(c, c, 7, padding=3, groups=c)\n        self.pw1, self.pw2 = nn.Conv2d(c, r * c, 1), nn.Conv2d(r * c, c, 1)\n        self.grn = GRN(r * c)\n        self.ada = nn.Sequential(nn.SiLU(), nn.Linear(c, 3 * c))\n        nn.init.zeros_(self.ada[1].weight)              # Start the condition branch at zero\n        nn.init.zeros_(self.ada[1].bias)\n\n    def forward(self, x, cond):\n        shift, scale, gate = self.ada(cond)[:, :, None, None].chunk(3, dim=1)\n        h = F.layer_norm(self.dw(x).permute(0, 2, 3, 1), (x.shape[1],)).permute(0, 3, 1, 2)\n        h = h * (1 + scale) + shift                     # The condition determines channel scaling and shifting\n        h = self.pw2(self.grn(F.gelu(self.pw1(h))))\n        return x + gate * h                             # The gate is condition-dependent too\n\ntorch.manual_seed(0)\nc = 32\nblock = FCDMBlock(c)\nx = torch.randn(2, c, 16, 16)\ncond_a, cond_b = torch.randn(1, c).expand(2, c), torch.randn(1, c).expand(2, c)   # Two different class-plus-time embeddings\n\nwith torch.no_grad():\n    print(\"At initialization: maximum output-input difference\", f\"{(block(x, cond_a) - x).abs().max():.1e}\")\n    nn.init.normal_(block.ada[1].weight, std=0.1)    # Simulate weights after some training\n    ya, yb = block(x, cond_a), block(x, cond_b)\n    print(\"After training: output difference for the same input under different conditions\", f\"{(ya - yb).abs().max():.2f}\")\n```\n\n```text\nAt initialization: maximum output-input difference 0.0e+00\nAfter training: output difference for the same input under different conditions 0.85\n```\n\nAt initialization, the block leaves the input completely unchanged. Once the condition branch's weights develop, the same input is processed differently under different classes and timesteps. These three parameter sets inject the condition.\n\n### A U Shape with Only Two Knobs\n\nThe blocks form a U. Descending through three levels, channels are $C$, $2C$, and $4C$, with $L$, $2L$, and $4L$ blocks, doubling both at each downsampling. The ascent is symmetric, with skips between matching levels. The whole architecture has only two hyperparameters, $L$ and $C$.\n\nFor a fair DiT comparison, all four sizes match DiT's parameter counts:\n\n| Model | $L$ | $C$ | Parameters | FLOPs | FLOPs relative to DiT |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| FCDM-S | 2 | 128 | 32.7M | 3.1G | 50.8% |\n| FCDM-B | 2 | 256 | 127.7M | 12.2G | 53.0% |\n| FCDM-L | 2 | 512 | 504.5M | 48.3G | 59.9% |\n| FCDM-XL | 3 | 512 | 698.8M | 64.6G | 54.5% |\n\n*Table: from the paper's Table 2; FLOPs at 256×256 resolution.*\n\nAt matched parameters, FLOPs are roughly half of DiT's.\n\n### How It Differs from DiCo\n\nDiCo was the strongest preceding convolutional diffusion model, with a structure resembling FCDM's. The authors devote a section to their differences.\n\nDiCo's convolution module keeps the original channel width throughout. FCDM uses ConvNeXt's inverted bottleneck: depthwise convolution first at the original width, then channel expansion. Its depthwise cost therefore stays independent of expansion ratio. DiCo adds a 1×1 convolution for channel attention to diversify channel responses; FCDM achieves a similar effect with GRN at almost no extra parameter cost. DiCo also has a separate feed-forward module, whose job FCDM's inverted bottleneck already handles, so FCDM omits it.\n\nAt matched parameters, FCDM consequently takes roughly three quarters of DiCo's FLOPs.\n\n## Discussion\n\n### Experimental Setup\n\nTraining follows DiT's pipeline exactly: AdamW, fixed learning rate $10^{-4}$, no weight decay, batch size 256, only horizontal-flip augmentation, and EMA weights. Evaluation uses FID, a distance between feature distributions of 50,000 generated images and real images; lower is better.\n\n### Beating DiT at Every Size\n\nFirst, results after 400,000 training steps, without classifier-free guidance:\n\n| Model | FLOPs | Inference throughput | FID |\n| --- | ---: | ---: | ---: |\n| DiT-S/2 | 6.1G | 1234 | 68.4 |\n| FCDM-S | 3.1G | 2687 | **48.5** |\n| DiT-B/2 | 23.0G | 380 | 43.5 |\n| FCDM-B | 12.2G | 1002 | **26.2** |\n| DiT-L/2 | 80.7G | 115 | 23.3 |\n| FCDM-L | 48.3G | 381 | **13.8** |\n| DiT-XL/2 | 118.6G | 81 | 19.5 |\n| FCDM-XL | 64.6G | 273 | **10.7** |\n| DiT-XL/2 (7 million steps) | 118.6G | 81 | 9.6 |\n| FCDM-XL (1 million steps) | 64.6G | 273 | **7.9** |\n\n*Table: from the paper's Tables 1 and 3. Throughput uses the paper's reported unit, it/s.*\n\nAt the same step count, FCDM has much lower FID at every size and 2.2 to 3.4 times DiT's inference throughput. The last two rows show convergence most clearly: FCDM-XL at 1 million steps achieves lower FID than DiT-XL/2 at 7 million.\n\nAgainst other convolutional backbones, the gap is much smaller. At 400,000 steps, DiCo-L gets FID 13.66, slightly better than FCDM-L's 13.83. At XL, FCDM gets 10.72 against DiCo-XL's 11.67, with one quarter fewer FLOPs. For small models, DiC has the highest throughput, using standard 3×3 convolutions with strong hardware optimization.\n\n### With Guidance\n\nTrain FCDM-XL for 2 million steps, or 400 epochs, add classifier-free guidance, and compare published results:\n\n| Model | Training epochs | FLOPs | FID | IS |\n| --- | ---: | ---: | ---: | ---: |\n| DiT-XL/2 | 1400 | 118.6G | 2.27 | 278.2 |\n| SiT-XL/2 | 1400 | 118.6G | 2.06 | 277.5 |\n| DiCo-XL | 750 | 87.3G | 2.05 | 282.2 |\n| FCDM-XL | 400 | 64.6G | **2.03** | **285.7** |\n\n*Table: selected rows from the paper's Table 4, ImageNet 256×256.*\n\nFID 2.03 improves on DiT-XL/2's 2.27 with fewer than one third the training epochs. Against SiT and DiCo, though, the difference is only in the second decimal place. The authors explicitly say that FCDM beats comparable models under DiT's experimental setup while still trailing the latest leading results from EDM-2 and Simpler Diffusion.\n\n### Double the Resolution\n\nAt 512×512, FCDM-XL gets FID 7.46 after 1 million steps; DiT-XL/2 gets 12.03 after 3 million, and DiCo-XL gets 8.10 after 1.3 million.\n\nThe change in speed is more interesting. From resolution 256 to 512, DiT-XL/2 throughput falls from 80.5 to 18.6, a drop by more than a factor of four. FCDM-XL falls from 272.7 to 129.6, roughly halving. Attention cost grows quadratically with token count, while convolution grows linearly with pixel count, widening the gap at higher resolutions.\n\n### Ablation\n\nTrain FCDM-L for 200,000 steps, change one component at a time, and compare FID:\n\n| Change | FID |\n| --- | ---: |\n| FCDM-L (7×7 depthwise, GRN, inverted bottleneck) | **19.97** |\n| Change depthwise kernel to 5×5 | 20.48 |\n| Change depthwise kernel to 3×3 | 21.28 |\n| Replace GRN with DiCo's channel attention | 23.85 |\n| Add DiCo's feed-forward module | 28.52 |\n| Remove inverted bottleneck | 28.76 |\n| Use ResNet blocks | 31.14 |\n\n*Table: from the paper's Table 6. The final four rows adjust channels to match FLOPs.*\n\nSmaller kernels perform worse: large kernels also help generation. Transferring DiCo's designs hurts, and removing the inverted bottleneck or returning to ResNet blocks hurts much more. Almost every part of ConvNeXt's design holds up for generation.\n\n### Four Graphics Cards\n\nThe paper emphasizes training cost. On four RTX 4090s, FCDM-XL with gradient checkpointing and batch size 256 runs at roughly 0.9 steps per second. Batch size 256 also fits on one 40GB A100.\n\nAt that speed, 400,000 steps take about 5 days, and 2 million take about 26 days. These are estimates calculated in this note from the paper's speed figure. This is a scale an ordinary lab can reach.\n\nThe official repository provides training and sampling code under CC BY-NC, with no pretrained weights currently supplied.\n\n## Conclusion\n\nFCDM invents almost nothing new. ConvNeXt V2 blocks, DiT's adaLN-Zero conditioning, and a U-shaped architecture with just two knobs are all existing components. Combined, they use half of DiT's FLOPs, converge several times faster, and gain more at higher resolutions.\n\nThe ablations are more persuasive than 2.03. Adding DiCo's refinements hurts, and removing ConvNeXt's design choices hurts too. A block designed for classification transfers almost unchanged to generation.\n\nFor the question in the subtitle, at least for ImageNet class-conditional generation, a Transformer is optional.\n\nText-to-image is still early. The appendix trains on MS-COCO for just 100,000 steps and inspects a few generated images, conditioning only on pooled CLIP text vectors. Accepting full text sequences as SD3 does is explicitly left for future work.\n","imageSizes":{"/content/papers/assets/2a98fdd408b56d724e2b57d3c4a4a2e3c8f3d1ffc8384c400f1ffceff0f84aad.png":{"width":1960,"height":1280}}}