{"notes":[{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"调好再比","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/classic-gnns.b7baf827a126.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三个经典模型","text":"三个经典模型","depth":3},{"id":"四个超参数","text":"四个超参数","depth":3},{"id":"实验设置","text":"实验设置","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"同配图","text":"同配图","depth":3},{"id":"异配图","text":"异配图","depth":3},{"id":"大图","text":"大图","depth":3},{"id":"哪个超参数最关键","text":"哪个超参数最关键","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/deeper-insights","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Deeper Insights","subtitle":"抹得太平了","author":"Z. Yuan","paperDate":"2018-01-22","publishedOn":"2023-11-29","paperTitle":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","paperUrl":"https://arxiv.org/abs/1801.07606","bibliography":{"title":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","authors":["Qimai Li","Zhichao Han","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/1801.07606","pdfUrl":"https://arxiv.org/pdf/1801.07606v1","codeUrl":null,"preprint":{"id":"1801.07606","submittedOn":"2018-01-22","url":"https://arxiv.org/abs/1801.07606"},"publication":{"venue":"AAAI","year":2018,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/11604"},"citation":"@misc{graphlearningdeeperinsights2018,\n  title = {{Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning}},\n  author = {Qimai Li and Zhichao Han and Xiao-Ming Wu},\n  year = {2018},\n  eprint = {1801.07606},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.07606}\n}\n","citationPath":"/content/papers/citations/graph-learning/deeper-insights.52354c86fac9.bib"},"thumbnail":{"src":"/content/papers/assets/2a13880b48fae6456e19c622cee1d7c7907f8c31d6890722d4980a381fddb06a.png","width":1396,"height":268},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/deeper-insights.3dfb15278376.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"先和全连接网络比","text":"先和全连接网络比","depth":3},{"id":"拉普拉斯平滑","text":"拉普拉斯平滑","depth":3},{"id":"平滑太多次","text":"平滑太多次","depth":3},{"id":"太浅也不行","text":"太浅也不行","depth":3},{"id":"补救让-gcn-多看到一些标签","text":"补救：让 GCN 多看到一些标签","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/appnp","relation":"referenced","topic":""},{"id":"graph-learning/sgc","relation":"referenced","topic":""}]},{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"老骥伏枥","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/gnn-plus.15890d5dd1bb.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"两种任务","text":"两种任务","depth":3},{"id":"比较公不公平","text":"比较公不公平","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三个经典模型","text":"三个经典模型","depth":3},{"id":"六个技巧","text":"六个技巧","depth":3},{"id":"怎么比","text":"怎么比","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"整体结果","text":"整体结果","depth":3},{"id":"也比较快","text":"也比较快","depth":3},{"id":"每个技巧各管什么","text":"每个技巧各管什么","depth":3},{"id":"作者自己说的限制","text":"作者自己说的限制","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]}],"text":"Graph Transformer 先在分子这类小图上站稳，接着有人把它推向节点分类，尤其是动辄上百万个节点的大图。NodeFormer、SGFormer、Polynormer 这些模型的论文里，GCN、GAT、GraphSAGE 常常被甩在后面。\n\n北京航空航天大学的 Yuankai Luo、Lei Shi 与香港理工大学的 Xiao-Ming Wu 注意到一件事。这些新模型里，不少明着或暗着，还是靠一个 GNN 去学局部的表示，再搭配全局注意力。\n\n那么，传统的消息传递在节点分类上，是不是被低估了？\n\nWu 对这个问题并不陌生。2018 年，他和 Qimai Li 等人把 GCN 解释成拉普拉斯平滑，也指出叠太深会 over-smoothing。\n\n\n> [!TIP]\n> \n> GCN 为什么有效、又为什么叠不深，可以参考我们的笔记：\n> \n> - **[[18.01] Deeper Insights：抹得太平了](/blog/papers/graph-learning/deeper-insights)**\n\n\n## 定义问题\n\n节点分类是在一张图上，用少数已知标签的节点，预测其他节点的类别。图可以分成两种：\n\n- **同配图**（homophily）：相连的节点多半同类，例如互相引用的论文。\n- **异配图**（heterophily）：相连的节点常常不同类。\n\n一般认为 GNN 隐含了同配性的假设，所以在异配图上表现不好；但近年已有研究发现，标准的 GCN 在异配图上其实也不差。\n\n过去也有人重新评估 GNN，作者认为那些研究的数据集太少、种类不够多，超参数也没有调完整。这篇要做一次规模够大的检查，看经典的 GNN 只调超参数，能走到哪里。\n\n## 解决问题\n\n### 三个经典模型\n\n主角是三个最基本的 GNN。GCN 照度数加权平均自己和邻居；GraphSAGE 把自己和邻居的平均分开乘权重；GAT 用注意力决定每个邻居的份量。模型本身一点都不改。\n\n### 四个超参数\n\n要调的，是几个在其他神经网络上早就用得很习惯的东西：\n\n- **归一化**：在每一层的激活函数之前加 BatchNorm 或 LayerNorm。\n- **Dropout**：加在激活函数之后。\n- **残差连接**：把上一层的表示乘上一个权重矩阵 $W_r$，直接加进这一层：\n\n$$\nh_v^{l}=\\mathrm{Dropout}\\Big(\\sigma\\Big(\\mathrm{Norm}\\Big(h_v^{l-1}W_r^l+\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l\\Big)\\Big)\\Big)\n$$\n\n- **层数**：从 1 层试到 10 层。\n\n残差在异配图上特别有用。用一个极端的例子看，六个节点绕成一圈，相邻的节点一定不同类，每个节点的特征就是自己类别的 one-hot：\n\n```python\nimport numpy as np\n\nn = 6\nM = np.zeros((n, n))\nfor i in range(n):\n    M[i, (i + 1) % n] = M[(i + 1) % n, i] = 1      # 六个节点绕成一圈\nlabel = np.array([0, 1, 0, 1, 0, 1])               # 相邻的节点一定不同类\nX = np.eye(2)[label]                               # 特征就是自己的类别\n\nAt = M + np.eye(n)\nd = At.sum(1)\nA_hat = At / np.sqrt(np.outer(d, d))               # GCN 的加权平均\n\ngcn = A_hat @ X                                    # 只有邻居加权平均\nres = X + A_hat @ X                                # 再加上自己原本的表示（W_r 取单位矩阵）\nprint(\"节点 0，GCN 一层后:\", gcn[0].round(2), \" 猜对\", (gcn.argmax(1) == label).sum(), \"个\")\nprint(\"节点 0，加上残差后:\", res[0].round(2), \" 猜对\", (res.argmax(1) == label).sum(), \"个\")\n```\n\n```text\n节点 0，GCN 一层后: [0.33 0.67]  猜对 0 个\n节点 0，加上残差后: [1.33 0.67]  猜对 6 个\n```\n\n每个节点有两个异类的邻居，平均之后，邻居的类别盖过了自己；加上残差，自己原本的信息留了下来。这只是示意，实际的 $W_r$ 是学出来的。\n\n### 实验设置\n\n数据集一共 18 个：\n\n- **同配图** 8 个：Cora、CiteSeer、PubMed、Computer、Photo、CS、Physics、WikiCS。\n- **异配图** 6 个：Squirrel、Chameleon、Amazon-Ratings、Roman-Empire、Minesweeper、Questions。\n- **大图** 4 个：OGB 的 ogbn-proteins、ogbn-arxiv、ogbn-products，以及社交网络 pokec，节点数从 13 万到 245 万。\n\n比较的对象包括 SGFormer、Polynormer、GOAT、NodeFormer、NAGphormer 这些为大图设计的 Graph Transformer，以及 GraphGPS、Exphormer；异配图上另外加了 H2GCN 等专为异配图设计的 GNN。\n\n超参数的搜索范围沿用 Polynormer，学习率取 {0.001, 0.005, 0.01}，隐藏维度取 {64, 256, 512}，再决定要不要用归一化（BN 或 LN）和残差，dropout 取 {0.2, 0.3, 0.5, 0.7}，层数从 1 到 10，最多训练 2,500 个 epoch。作者也用同样的搜索范围和训练环境，重新训练了表中的 Graph Transformer；每个结果是 5 次不同初始化的平均。\n\n## 讨论\n\n18 个数据集里，调过的经典 GNN 在 17 个排第一。下面分三类看。\n\n### 同配图\n\n调参之后，GCN 在 Cora 从 81.60% 升到 85.10%，在 Computer 从 89.65% 升到 93.99%。GAT 在 WikiCS 上升了 4.16 个百分点到 81.07%，从第七名跳到第一，超过原本领先的 Polynormer。Cora、CiteSeer、PubMed、Physics 上，第一名都是调过的 GCN。\n\n### 异配图\n\n异配图上的进步更大。GCN 在 Squirrel 和 Chameleon 上分别升了 6.34 和 4.98 个百分点，都排第一；Minesweeper 和 Questions 也是 GCN 第一。H2GCN、GloGNN 这些专为异配图设计的模型，反而都排在后面。\n\n最夸张的是 Roman-Empire，GCN 从 73.69% 升到 91.27%，多了 17.58 个百分点。这也是 18 个数据集里唯一的例外，Polynormer 的 92.66% 仍然最高。\n\n### 大图\n\n| 模型 | ogbn-proteins（ROC-AUC） | ogbn-arxiv | ogbn-products | pokec |\n| --- | ---: | ---: | ---: | ---: |\n| GraphGPS | 77.15 | 71.23 | 内存不足 | 内存不足 |\n| Exphormer | 77.62 | 72.32 | 内存不足 | 内存不足 |\n| SGFormer | 79.92 | 72.76 | 81.54 | 82.44 |\n| Polynormer | 79.53 | 73.40 | 83.82 | 86.06 |\n| GCN（调参后） | 77.29 | **73.53** | 82.33 | **86.33** |\n| GraphSAGE（调参后） | 82.21 | 73.00 | **83.89** | 85.97 |\n| GAT（调参后） | **85.01** | 73.30 | 80.99 | 86.19 |\n\n表中的 Graph Transformer 都是作者用同样条件重新训练的结果。245 万个节点的 ogbn-products 和 163 万个节点的 pokec 上，GraphGPS 和 Exphormer 都内存不足。四个大图上，第一名都是经典 GNN。pokec 上的 GCN 比原本回报的数字多了 10.88 个百分点，ogbn-proteins 上的 GAT 多了 12.99 个百分点。\n\n作者的解读是，消息传递在大图上学节点表示依然非常有效，现有的 Graph Transformer 也还没有真正解决 over-smoothing 和长距离依赖这些 GNN 的老问题。\n\n### 哪个超参数最关键\n\n作者把四个超参数一个一个拿掉，看准确率掉多少，整理出四点：\n\n1. **归一化在大图上很重要**，小图上影响不大。ogbn-proteins 上拿掉归一化，GraphSAGE 和 GAT 分别掉 4.79 和 4.69 个百分点。作者推测，大图的节点特征分布差异大，归一化让训练时的分布比较稳定。\n2. **Dropout 几乎总是有帮助**。拿掉它，GraphSAGE 在 PubMed 掉 2.70、在 Roman-Empire 掉 6.57 个百分点。\n3. **残差在异配图上影响最大**。Roman-Empire 上拿掉残差，GCN 掉了 16.43 个百分点，从 91.27% 掉回 74.84%，GCN 在这个数据集的进步几乎都来自它。\n4. **异配图通常需要更深的网络**。\n\n[![原论文 Figure 1：六张折线图，横轴是层数 2 到 10。上排是 GCN，下排是 GraphSAGE；左到右分别是同配图（Computer、Photo、CS）、异配图（Roman-Empire、Minesweeper、Questions）与大图（ogbn-arxiv）。同配图的准确率在 6 层附近持平或随层数下降；异配图的三条线大致随层数上升；ogbn-arxiv 在 4 到 5 层达到最高后下降。](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)\n\n*原论文 Figure 1（Luo 等人），取自 [**PDF 第 9 页**](https://arxiv.org/pdf/2406.08993v2#page=9)。图注未收录。*\n\n上排是 GCN，下排是 GraphSAGE。中间那列的异配图，三条线大致随着层数往上走；左边的同配图，多数在 6 层之前就到顶了。作者认为，异配图的结构比较复杂，需要更深的网络才能抓到节点之间的关系，而深网络要训练得起来，就得靠残差连接。\n\n## 结论\n\n这篇论文没有提出新模型，只把三个最老的 GNN 好好调了一次。结果在 18 个数据集里，它们有 17 个拿第一；在最大的两张图上，有些 Graph Transformer 甚至跑不起来。\n\n隔年，同一组作者把同样的做法搬到图层级的任务，替 GCN、GIN、GatedGCN 加上六个常见的技巧，再和 Graph Transformer 比了一次。\n\n\n> [!TIP]\n> \n> 图层级上的重新比较，可以参考我们的笔记：\n> \n> - **[[25.02] GNN+：老骥伏枥](/blog/papers/graph-learning/gnn-plus)**\n\n\n新的 Graph Transformer 在节点分类上大胜 GCN 时，先翻附录看看 GCN 有没有加残差和归一化、层数试到几层，再决定要不要相信那张表。\n","imageSizes":{"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png":{"width":1080,"height":516}}}