{"notes":[{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"老骥伏枥","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/gnn-plus.15890d5dd1bb.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"两种任务","text":"两种任务","depth":3},{"id":"比较公不公平","text":"比较公不公平","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三个经典模型","text":"三个经典模型","depth":3},{"id":"六个技巧","text":"六个技巧","depth":3},{"id":"怎么比","text":"怎么比","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"整体结果","text":"整体结果","depth":3},{"id":"也比较快","text":"也比较快","depth":3},{"id":"每个技巧各管什么","text":"每个技巧各管什么","depth":3},{"id":"作者自己说的限制","text":"作者自己说的限制","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]},{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"调好再比","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/classic-gnns.b7baf827a126.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三个经典模型","text":"三个经典模型","depth":3},{"id":"四个超参数","text":"四个超参数","depth":3},{"id":"实验设置","text":"实验设置","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"同配图","text":"同配图","depth":3},{"id":"异配图","text":"异配图","depth":3},{"id":"大图","text":"大图","depth":3},{"id":"哪个超参数最关键","text":"哪个超参数最关键","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"图上的 GPS","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/graphgps.74aedb083e17.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"位置与结构","text":"位置与结构","depth":3},{"id":"为什么消息传递需要它们","text":"为什么消息传递需要它们","depth":3},{"id":"先传递再注意会先丢掉信息","text":"先传递再注意，会先丢掉信息","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三份材料","text":"三份材料","depth":3},{"id":"一层-gps","text":"一层 GPS","depth":3},{"id":"线性的复杂度","text":"线性的复杂度","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"消融两条路各自的份量","text":"消融：两条路各自的份量","depth":3},{"id":"和其他模型比","text":"和其他模型比","depth":3},{"id":"一年后的重新评估","text":"一年后的重新评估","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/where-did-the-gap-go","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Where Did the Gap Go?","subtitle":"差距去哪了？","author":"Z. Yuan","paperDate":"2023-09-01","publishedOn":"2025-11-12","paperTitle":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","paperUrl":"https://arxiv.org/abs/2309.00367","bibliography":{"title":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","authors":["Jan Tönshoff","Martin Ritzert","Eran Rosenbluth","Martin Grohe"],"url":"https://arxiv.org/abs/2309.00367","pdfUrl":"https://arxiv.org/pdf/2309.00367v2","codeUrl":"https://github.com/toenshoff/LRGB","preprint":{"id":"2309.00367","submittedOn":"2023-09-01","url":"https://arxiv.org/abs/2309.00367"},"publication":{"venue":"Transactions on Machine Learning Research","shortVenue":"TMLR","year":2024,"kind":"journal","url":"https://openreview.net/forum?id=Nm0WX86sKv"},"citation":"@misc{graphlearningwheredidthegapgo2023,\n  title = {{Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark}},\n  author = {Jan Tönshoff and Martin Ritzert and Eran Rosenbluth and Martin Grohe},\n  year = {2023},\n  eprint = {2309.00367},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2309.00367}\n}\n","citationPath":"/content/papers/citations/graph-learning/where-did-the-gap-go.288771073197.bib"},"thumbnail":{"src":"/content/papers/assets/2381a94a5409a041eaab5b06f8dd33ebdd448087d689d89d780486cf3b900353.png","width":1118,"height":416},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/where-did-the-gap-go.ff47e75698ca.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"基本的超参数搜索","text":"基本的超参数搜索","depth":3},{"id":"逐通道归一化","text":"逐通道归一化","depth":3},{"id":"filtered-mrr","text":"filtered MRR","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"肽数据集","text":"肽数据集","depth":3},{"id":"一个预测头","text":"一个预测头","depth":3},{"id":"超像素数据集","text":"超像素数据集","depth":3},{"id":"pcqm-contact","text":"PCQM-Contact","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/lrgb","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]}],"text":"GraphGPS 之后，图分类、图回归这类图层级任务的排行榜，几乎被 Graph Transformer 占满。论文里的说法也大致一致，消息传递的 GNN 有表达力上限，叠深会 over-smoothing，传远会 over-squashing，全局注意力正好补上这些缺口。\n\n北京航空航天大学与香港理工大学的 Yuankai Luo、Lei Shi、Xiao-Ming Wu 不太相信。前一年，他们在节点分类上做过一次，把 GCN、GAT、GraphSAGE 放进和新模型一样大的超参数搜索空间，结果在 18 个数据集里的 17 个打平甚至超过当时最好的模型。\n\n\n> [!TIP]\n> \n> 节点分类上的那次重新比较，可以参考我们的笔记：\n> \n> - **[[24.06] Classic GNNs：调好再比](/blog/papers/graph-learning/classic-gnns)**\n\n\n这篇换到图层级的任务，问同一个问题。经典的 GNN 只要配上现在常用的训练技巧，能不能和 Graph Transformer 一较高下？\n\n## 定义问题\n\n### 两种任务\n\n作者先把节点层级和图层级的任务分清楚。节点分类通常是一张大图，例如社交网络或引用网络，训练、验证、测试是同一张图上不同的节点。图层级的任务则以整张图为单位，数据集是一大堆小图，例如分子或蛋白质，切分的也是图。\n\n两者的数据组成、切分方式、训练策略都不同，在节点分类上成立的结论，不能直接搬到图层级。\n\n### 比较公不公平\n\n文献里，Graph Transformer 胜过 GNN 的证据，大多来自图层级的任务，尤其是小分子的分类与回归。但这些比较里的 GNN 基准，常常是好几年前的设置，没有用上后来新模型习以为常的技巧。\n\n作者想知道的是，差距有多少来自架构本身，又有多少来自基准没有调好。\n\n## 解决问题\n\n### 三个经典模型\n\n作者挑了三个消息传递模型当骨架：\n\n- **GCN**：邻居和自己依度数做对称归一化后加总，再乘权重。\n- **GIN**：自己乘上 $1+\\epsilon$，加上邻居总和，再过一个 MLP。\n- **GatedGCN**：在聚合时，每条边有一个由两端节点与边特征算出的闸门，决定这个邻居的消息能通过多少。\n\n### 六个技巧\n\n再把六个常见的技巧加上去，称为 GNN+：\n\n1. **边特征**：把边的特征 $e_{uv}$ 乘上一个可学的矩阵，加进消息传递里。分子图的化学键、超像素图的空间关系，都在边上。\n2. **归一化**：每一层聚合之后、激活函数之前，加上 BatchNorm，稳定各层的分布。\n3. **Dropout**：加在激活函数之后，减少神经元之间的共同适应。\n4. **残差连接**：把这一层的输入直接加到输出上。\n5. **前馈网络（FFN）**：每层再接一个两层的 FFN，和 Transformer 里的那一块一样，也带残差与归一化。\n6. **位置编码**：把随机游走结构编码（RWSE），也就是从节点出发走 $m$ 步回到自己的概率，接在输入特征后面，再过一个线性层。\n\n以 GCN 为例，加上全部技巧之后，一层 GCN+ 写成：\n\n$$\nh_v^l=\\mathrm{FFN}\\left(\\mathrm{Dropout}\\left(\\sigma\\left(\\mathrm{BN}\\left(\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l+e_{uv}W_e^l\\right)\\right)\\right)+h_v^{l-1}\\right)\n$$\n\n$$\n\\mathrm{FFN}(h)=\\mathrm{BN}\\left(\\sigma\\left(hW_{\\mathrm{FFN}_1}^l\\right)W_{\\mathrm{FFN}_2}^l+h\\right)\n$$\n\n从内到外读，邻居消息加上边特征，接 BatchNorm、ReLU、dropout，再加上一层的输入当残差，最后过 FFN。\n\n[![原论文 Figure 2：GNN+ 的架构。节点特征先接上位置编码，连同边特征送进重复 L 次的区块；每个区块是 GNN 层接 FFN，最后做 readout。右侧展开 GNN 层的内部，由下往上依序是消息传递、归一化、激活与 dropout、残差连接；FFN 内部则是 MLP 接 Add & Norm。](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)\n\n*原论文 Figure 2（Luo 等人），取自 [**PDF 第 3 页**](https://arxiv.org/pdf/2502.09263v3#page=3)。图注未收录。*\n\n这六样东西没有一样是新的。残差、归一化、dropout 来自 CNN；FFN 和位置编码来自 Transformer，RWSE 则是 GraphGPS 等 Graph Transformer 常用的结构编码。作者的贡献是把它们系统地组合起来，再做一次公平的比较。\n\n### 怎么比\n\n14 个图层级数据集，分成三组：\n\n- **Benchmarking GNNs**：ZINC（分子回归）、MNIST 与 CIFAR10（图像转成超像素图后分类）、PATTERN 与 CLUSTER（合成图上的节点分类）。\n- **LRGB**：Peptides-func、Peptides-struct（肽分子）、PascalVOC-SP、COCO-SP（超像素图上的语义分割），加上 MalNet-Tiny（恶意程序的调用图）。\n- **OGB**：ogbg-molhiv、ogbg-molpcba、ogbg-ppa、ogbg-code2。\n\n超参数的搜索空间比照 GraphGPS，参数预算也照各基准的规定，例如 ZINC 约 50 万个参数。作者也用同样的搜索空间和训练环境，重新训练了作为基准的 Graph Transformer；其余基准取各自论文或排行榜上的数字，这些多半已经是调过的结果。每组实验跑 5 个随机数种子。\n\n## 讨论\n\n### 整体结果\n\n在 14 个数据集上，加强后的经典 GNN 每一个都排进前三，其中 8 个拿第一。挑几组数字看：\n\n| 数据集（指标） | 原版 GCN | GCN+ | GatedGCN+ | GraphGPS | 表中其他模型最佳 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| ZINC（MAE ↓） | 0.367 | 0.076 | 0.077 | 0.070 | 0.057（TIGT） |\n| CIFAR10（准确率 ↑） | 55.71% | 69.82% | **77.22%** | 72.30% | 76.85%（GRED） |\n| Peptides-func（AP ↑） | 0.6860 | **0.7261** | 0.7006 | 0.6535 | 0.7150（DRew） |\n| Peptides-struct（MAE ↓） | 0.2460 | **0.2421** | 0.2431 | 0.2500 | 0.2449（Graph ViT） |\n| PascalVOC-SP（F1 ↑） | 0.2078 | 0.3357 | 0.4263 | 0.4440 | 0.4561（GSSC） |\n\n几件事可以看：\n\n1. **原版和加强版差很多**。ZINC 上，GCN 的 MAE 从 0.367 降到 0.076，GIN 从 0.526 降到 0.065，同一个骨架，只是补上了六个技巧。\n2. **小分子上，差距几乎消失**。ZINC 上 GIN+ 的 0.065 已经优于 GraphGPS；最好的 Graph Transformer 仍然略胜一筹，但差距从原本的数倍缩到零点零几。\n3. **长距离的数据集上，经典 GNN 反而领先**。Peptides-func 上 GCN+ 是 0.7261，高于所有列出的 Graph Transformer 和 Graph Mamba。注意表中「原版 GCN」在 LRGB 上用的，已经是 Tönshoff 等人 2023 年调过参数的数字；LRGB 原论文里的 GCN 只有 0.5930。\n4. **不是每一项都赢**。PascalVOC-SP 上，GatedGCN+ 仍落后 GraphGPS 的 2023 年版本与 GSSC。\n\nOGB 上，GatedGCN+ 在四个数据集都排进前三，其中三个拿第一。ogbg-ppa 上提升约 9%，在 OGB 排行榜上排第一；ogbg-molhiv 和 ogbg-molpcba 上，它和先在其他数据集预训练过的 Graphormer、EGT 表现相当。\n\n### 也比较快\n\n经典 GNN 没有全局注意力，计算量小得多。以每个 epoch 的训练时间来看，ZINC 上 GraphGPS 要 21 秒，GCN+ 要 7 秒；MalNet-Tiny 的图平均有 1,410 个节点，GraphGPS 要 46 秒，GCN+ 只要 6 秒。\n\n### 每个技巧各管什么\n\n作者对每个模型逐一拿掉六个技巧，整理出六点观察：\n\n1. **边特征在分子与超像素图上特别重要**。分子的边是化学键，超像素的边是空间关系；代码的调用图上，边特征和任务比较无关，拿掉影响不大。\n2. **归一化在大数据集上影响比较大**。ogbg-ppa 有 15 万张图，拿掉归一化，三个模型的准确率都掉了约 15%。\n3. **Dropout 几乎都有帮助，但只要一点点**。最佳的 dropout 率有 97% 不超过 0.2，64% 不超过 0.1。作者指出这和他们在节点分类上的观察相反，那边通常需要比较高的 dropout。\n4. **残差连接大多不可或缺**，只有在小图上的浅层网络例外。肽数据集只用 3 到 5 层，拿掉残差影响不大；其他数据集用了 10 层以上，拿掉残差常常大幅退步。\n5. **FFN 对 GCN+ 和 GIN+ 很关键**。MNIST 上，GIN+ 拿掉 FFN，准确率从 98.29% 掉到 11.35%，相对下降 88%。GatedGCN+ 本身有闸门，对 FFN 的依赖就小得多。\n6. **位置编码在小数据集上有效，大数据集上作用不大**。ZINC、PATTERN、CLUSTER、Peptides-func、ogbg-molhiv 只有一到四万张图，拿掉 RWSE 明显退步；超过十万张图的数据集，影响就不明显了。作者推测，数据够多时，模型比较不需要靠编码来理解图的结构。\n\n消融表里最夸张的是 CLUSTER。这是一个 6 类的节点分类任务，乱猜大约是 16.7%；三个模型只要拿掉残差连接，准确率都掉到 16.5% 到 16.8% 之间，GCN+ 拿掉归一化也一样。在这种 10 层以上的深网络里，少了残差或归一化，模型等于没学到东西。\n\n\n> [!TIP]\n> \n> 被拿来比较的 GraphGPS、表中 LRGB 基准所依据的重新评估，以及这篇用到的 GCN、GIN，可以参考我们的笔记：\n> \n> - **[[22.05] GraphGPS：图上的 GPS](/blog/papers/graph-learning/graphgps)**\n> - **[[23.09] Where Did the Gap Go?：差距去哪了？](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[16.09] GCN：一个参数够吗？](/blog/papers/graph-learning/gcn)**\n> - **[[18.10] GIN：加总才算数](/blog/papers/graph-learning/gin)**\n\n\n### 作者自己说的限制\n\n作者承认，这些结论完全来自实验，没有理论解释经典 GNN 为什么能追上来；六个技巧之间如何互相影响、能不能真的缓解 over-smoothing 和 over-squashing，也还需要理论上的分析。\n\n他们也引用了 Bechler-Speicher 等人 2025 年的讨论，指出现有的图数据集，未必能反映真实世界图问题的复杂与多样。等更有挑战性、更贴近应用的基准出现，经典 GNN 和 Graph Transformer 的相对位置，可能又会改变。\n\n## 结论\n\n这篇把 GCN、GIN、GatedGCN 补上六个常见的技巧，在 14 个图层级数据集上和 Graph Transformer 正面比较。结果全部排进前三，8 个第一，训练还快上好几倍。\n\n读这篇最好连同它的前作与 Tönshoff 等人的重新评估一起看。三篇都在说，一个新架构的优势，得在基准也调到同样好之后才算数。\n\n注意力也没有因此退场。GNN+ 里的 FFN 和 RWSE，本来就是 Graph Transformer 那一系常用的组件；两条路线的工具箱，已经混在一起了。\n\n要在图层级的任务上试新模型，先把 GatedGCN+ 跑起来当基准吧。作者把[代码](https://github.com/LUOyk1999/GNNPlus)和每个数据集的超参数都公开了，花不了多少时间。\n","imageSizes":{"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png":{"width":604,"height":510}}}