{"notes":[{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"图上的 GPS","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/graphgps.74aedb083e17.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"位置与结构","text":"位置与结构","depth":3},{"id":"为什么消息传递需要它们","text":"为什么消息传递需要它们","depth":3},{"id":"先传递再注意会先丢掉信息","text":"先传递再注意，会先丢掉信息","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"三份材料","text":"三份材料","depth":3},{"id":"一层-gps","text":"一层 GPS","depth":3},{"id":"线性的复杂度","text":"线性的复杂度","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"消融两条路各自的份量","text":"消融：两条路各自的份量","depth":3},{"id":"和其他模型比","text":"和其他模型比","depth":3},{"id":"一年后的重新评估","text":"一年后的重新评估","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/graph-transformer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graph Transformer","subtitle":"连满反而糟","author":"Z. Yuan","paperDate":"2020-12-17","publishedOn":"2024-03-13","paperTitle":"A Generalization of Transformer Networks to Graphs","paperUrl":"https://arxiv.org/abs/2012.09699","bibliography":{"title":"A Generalization of Transformer Networks to Graphs","authors":["Vijay Prakash Dwivedi","Xavier Bresson"],"url":"https://arxiv.org/abs/2012.09699","pdfUrl":"https://arxiv.org/pdf/2012.09699v2","codeUrl":"https://github.com/graphdeeplearning/graphtransformer","preprint":{"id":"2012.09699","submittedOn":"2020-12-17","url":"https://arxiv.org/abs/2012.09699"},"publication":{"venue":"AAAI Workshop on Deep Learning on Graphs","shortVenue":"DLG-AAAI","year":2021,"kind":"workshop","url":"https://arxiv.org/abs/2012.09699v2"},"citation":"@misc{graphlearninggraphtransformer2020,\n  title = {{A Generalization of Transformer Networks to Graphs}},\n  author = {Vijay Prakash Dwivedi and Xavier Bresson},\n  year = {2020},\n  eprint = {2012.09699},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2012.09699}\n}\n","citationPath":"/content/papers/citations/graph-learning/graph-transformer.4e2f841bbbf0.bib"},"thumbnail":{"src":"/content/papers/assets/0540122812c6e6914f22240411cf2d172a94764bf8a0e6f6ab2cdae7c618f9fb.png","width":1324,"height":1000},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/graph-transformer.1ebf78897bd9.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"只在邻居之间算注意力","text":"只在邻居之间算注意力","depth":3},{"id":"拉普拉斯位置编码","text":"拉普拉斯位置编码","depth":3},{"id":"边特征与归一化","text":"边特征与归一化","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"实验设置","text":"实验设置","depth":3},{"id":"全连接会怎样","text":"全连接会怎样？","depth":3},{"id":"位置编码与-batchnorm","text":"位置编码与 BatchNorm","depth":3},{"id":"和-gnn-比","text":"和 GNN 比","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""}]},{"id":"graph-learning/san","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"SAN","subtitle":"听见图的形状","author":"Z. Yuan","paperDate":"2021-06-07","publishedOn":"2024-03-22","world":"external/san","paperTitle":"Rethinking Graph Transformers with Spectral Attention","paperUrl":"https://arxiv.org/abs/2106.03893","bibliography":{"title":"Rethinking Graph Transformers with Spectral Attention","authors":["Devin Kreuzer","Dominique Beaini","William L. Hamilton","Vincent Létourneau","Prudencio Tossou"],"url":"https://arxiv.org/abs/2106.03893","pdfUrl":"https://arxiv.org/pdf/2106.03893v3","codeUrl":"https://github.com/DevinKreuzer/SAN","preprint":{"id":"2106.03893","submittedOn":"2021-06-07","url":"https://arxiv.org/abs/2106.03893"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/b4fd1d2cb085390fbbadae65e07876a7-Abstract.html"},"citation":"@misc{graphlearningsan2021,\n  title = {{Rethinking Graph Transformers with Spectral Attention}},\n  author = {Devin Kreuzer and Dominique Beaini and William L. Hamilton and Vincent Létourneau and Prudencio Tossou},\n  year = {2021},\n  eprint = {2106.03893},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.03893}\n}\n","citationPath":"/content/papers/citations/graph-learning/san.e69e469c3b91.bib"},"thumbnail":{"src":"/content/papers/assets/239f3ac016f803fed4fedec5cad345c48bc6b97791249f0e5cfdf2efd433b5c2.png","width":1280,"height":840},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/san.2705d642b9c9.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"学出来的位置编码","text":"学出来的位置编码","depth":3},{"id":"全连接但分开两套注意力","text":"全连接，但分开两套注意力","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"gamma-该设多大","text":"\\gamma 该设多大？","depth":3},{"id":"全连接和-lpe-各自的贡献","text":"全连接和 LPE 各自的贡献","depth":3},{"id":"和当时最好的模型比","text":"和当时最好的模型比","depth":3},{"id":"代价","text":"代价","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""}]},{"id":"graph-learning/graphormer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graphormer","subtitle":"其实不差","author":"Z. Yuan","paperDate":"2021-06-09","publishedOn":"2024-03-28","world":"external/graphormer","paperTitle":"Do Transformers Really Perform Bad for Graph Representation?","paperUrl":"https://arxiv.org/abs/2106.05234","bibliography":{"title":"Do Transformers Really Perform Bad for Graph Representation?","authors":["Chengxuan Ying","Tianle Cai","Shengjie Luo","Shuxin Zheng","Guolin Ke","Di He","Yanming Shen","Tie-Yan Liu"],"url":"https://arxiv.org/abs/2106.05234","pdfUrl":"https://arxiv.org/pdf/2106.05234v5","codeUrl":"https://github.com/Microsoft/Graphormer","preprint":{"id":"2106.05234","submittedOn":"2021-06-09","url":"https://arxiv.org/abs/2106.05234"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/f1c1592588411002af340cbaedd6fc33-Abstract.html"},"citation":"@misc{graphlearninggraphormer2021,\n  title = {{Do Transformers Really Perform Bad for Graph Representation?}},\n  author = {Chengxuan Ying and Tianle Cai and Shengjie Luo and Shuxin Zheng and Guolin Ke and Di He and Yanming Shen and Tie-Yan Liu},\n  year = {2021},\n  eprint = {2106.05234},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.05234}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphormer.23b94dbe8ef3.bib"},"thumbnail":{"src":"/content/papers/assets/c2a2e0893f98e2ed6d93e1f4ee87040eeffbc63c4b955e6a319700c5e87f1847.png","width":872,"height":620},"locale":"zh-hans","path":"/content/papers/zh-hans/graph-learning/graphormer.3a73afc450c3.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"解决问题","text":"解决问题","depth":2},{"id":"中心性编码","text":"中心性编码","depth":3},{"id":"空间编码","text":"空间编码","depth":3},{"id":"边编码","text":"边编码","depth":3},{"id":"其他细节","text":"其他细节","depth":3},{"id":"能表示哪些-gnn","text":"能表示哪些 GNN？","depth":3},{"id":"比-wl-更强","text":"比 WL 更强","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"pcqm4m-lsc","text":"PCQM4M-LSC","depth":3},{"id":"其他数据集","text":"其他数据集","depth":3},{"id":"每个编码贡献多少","text":"每个编码贡献多少","depth":3},{"id":"代价","text":"代价","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""}]}],"text":"2021 年，Graphormer 用 Transformer 拿下 OGB-LSC 量子化学竞赛的冠军，Graph Transformer 一下子热了起来。\n\n道理不难理解。Transformer 让每个节点直接看所有节点，消息传递的几个老问题，叠深会 over-smoothing、传远会 over-squashing、分辨能力卡在 WL 测试，好像都能一口气绕过去。\n\n问题是，当时的 Graph Transformer 各做各的。有人用拉普拉斯的特征向量当位置编码，有人用最短路径距离当注意力的偏置，有人把消息传递层叠在前面；哪些设计真的有用、为什么有用，没有一个共同的说法。再加上全局注意力的成本是 $O(N^2)$，这些模型大多只能处理几百个节点的小图。\n\n\n> [!TIP]\n> \n> 只看邻居、以拉普拉斯特征向量当位置编码的 GT，全连接但分开两套注意力的 SAN，以及用最短路径距离当注意力偏置的 Graphormer，可以参考我们的笔记：\n> \n> - **[[20.12] Graph Transformer：连满反而糟](/blog/papers/graph-learning/graph-transformer)**\n> - **[[21.06] SAN：听见图的形状](/blog/papers/graph-learning/san)**\n> - **[[21.06] Graphormer：其实不差](/blog/papers/graph-learning/graphormer)**\n\n\nMila 的 Ladislav Rampášek、Mikhail Galkin 与南洋理工大学的 Vijay Prakash Dwivedi 等人想把这些整理成一份配方。标题里的 GPS，是 General、Powerful、Scalable 三个字的缩写。\n\n## 定义问题\n\n### 位置与结构\n\nTransformer 本身看不到图。把所有节点连在一起之后，谁和谁本来相邻，模型完全不知道；所以 Graph Transformer 都得额外提供图的信息。\n\n作者先把这些额外信息分成两类：\n\n- **位置编码（PE）**：告诉模型一个节点在图上的「位置」。图上相近的两个节点，PE 也应该相近。常见的是拉普拉斯矩阵的特征向量（LapPE）。\n- **结构编码（SE）**：描述节点或整张图的结构。两个节点周围的子图长得像，SE 就应该相近。常见的是随机游走矩阵 $m$ 次方的对角线（RWSE），也就是从自己出发、走 $m$ 步回到自己的概率。\n\n两类各自再分成局部、全局与相对三种，例如全局 SE 可以是拉普拉斯矩阵的特征值，相对 SE 可以是两个节点是否位在同一个子结构里。作者把当时的做法整理成一张表，并主张 PE 与 SE 最好以节点特征或真正的边来表示，才能搭配线性复杂度的注意力。\n\n### 为什么消息传递需要它们\n\n标准的消息传递网络，分辨能力和 1-WL 测试一样。作者用一组 Circular Skip Link 图说明这件事。两张不同构的图，WL 对每个节点都涂上同一个颜色，任何消息传递网络都分不出来；加上全局 PE，每个节点一开始就有独特的颜色，两张图就分得开；换成局部的 RWSE，也能抓到两张图跳接方式的差异。\n\n换句话说，这些信息无法单靠消息传递从图结构里学出来，得另外给。\n\n拿一个更小的例子，六个节点的六边形，和两个分开的三角形。两张图里每个节点都有两个邻居，WL 分不开。算一下 RWSE：\n\n```python\nimport numpy as np\nnp.set_printoptions(precision=3, suppress=True)\n\ndef graph(edges, n=6):\n    A = np.zeros((n, n))\n    for i, j in edges:\n        A[i, j] = A[j, i] = 1\n    return A\n\nhexagon = graph([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5), (5, 0)])\ntwo_triangles = graph([(0, 1), (1, 2), (2, 0), (3, 4), (4, 5), (5, 3)])\n\ndef rwse(A, steps=4):\n    P = A / A.sum(1, keepdims=True)                  # 随机游走的转移矩阵 D^-1 A\n    return np.stack([np.diag(np.linalg.matrix_power(P, m)) for m in range(1, steps + 1)], axis=1)\n\nprint(\"六边形      0 号节点:\", rwse(hexagon)[0])\nprint(\"两个三角形  0 号节点:\", rwse(two_triangles)[0])\n```\n\n```text\n六边形      0 号节点: [0.    0.5   0.    0.375]\n两个三角形  0 号节点: [0.    0.5   0.25  0.375]\n```\n\n差别在第三个数字，走 3 步回到自己的概率。三角形上有 0.25，六边形上是 0，因为六边形没有长度为 3 的环。作者也提到，$m$ 为奇数时，RWSE 可以看出节点是否在长度为 $m$ 的环上。\n\n### 先传递再注意，会先丢掉信息\n\n另一个常见的架构，是先叠几层消息传递，再接 Transformer，例如 GraphTrans。作者认为这样有风险。前面的消息传递层本身就有 over-smoothing、over-squashing 和表达力的限制，可能在一开始就把某些信息丢掉，后面的 Transformer 再强也救不回来。\n\n计算机视觉里「先卷积、再 Transformer」能成功，是因为网格上的卷积层表达力很强；作者不认为这套做法在图上能同样成立。\n\n## 解决问题\n\n### 三份材料\n\nGPS 的配方分三部分：\n\n1. **位置／结构编码**：在输入时加到节点特征，或边特征上。\n2. **局部的消息传递**：只沿着真正的边，可以处理边特征。\n3. **全局的注意力**：让每个节点看所有节点。\n\n每一部分都可以替换。消息传递可以用 GINE、GatedGCN 或 PNA，全局注意力可以用标准的 Transformer、Performer 或 BigBird。\n\n[![原论文 Figure 1：模块化的 GPS 架构。左侧三列分别是位置编码、结构编码与图特征，各自经过 MLP、DeepSet 或 SignNet 等处理后加到节点与边特征上；右侧是重复 L 次的 GPS 层，每层同时走「任一消息传递层（GINE、GatedGCN、PNA）」与「任一全局注意力（Transformer、Performer、BigBird）」，两者相加后接一个两层 MLP。](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)\n\n*原论文 Figure 1（Rampášek 等人），取自 [**PDF 第 3 页**](https://arxiv.org/pdf/2205.12454v4#page=3)。图注未收录。*\n\n### 一层 GPS\n\n每一层，两条路并行，结果相加后再过一个 MLP：\n\n$$\nX_M^{\\ell+1},\\,E^{\\ell+1}=\\mathrm{MPNN}_e^{\\ell}\\left(X^\\ell,E^\\ell,A\\right)\n$$\n\n$$\nX_T^{\\ell+1}=\\mathrm{GlobalAttn}^{\\ell}\\left(X^\\ell\\right)\n$$\n\n$$\nX^{\\ell+1}=\\mathrm{MLP}^{\\ell}\\left(X_M^{\\ell+1}+X_T^{\\ell+1}\\right)\n$$\n\n边特征只进消息传递那一条。式子里省略了残差连接和 BatchNorm，实际的实现两条路都有。\n\n接着上面的两张图，用随机权重写一层 GPS。消息传递用最简单的邻居总和，全局注意力用单头的 Transformer 注意力，最后把所有节点加总，当成整张图的表示：\n\n```python\nrng = np.random.default_rng(0)\nd = 4\nW_in = rng.normal(size=(4, d))                       # 把输入特征投影到 d 维\nWm, Wq, Wk, Wv = (rng.normal(size=(d, d)) for _ in range(4))\nW1, W2 = rng.normal(size=(d, 2 * d)), rng.normal(size=(2 * d, d))\n\ndef gps_layer(X, A):\n    X_M = X + np.maximum((A @ X) @ Wm, 0)            # 局部：邻居总和的消息传递，加残差\n    Q, K, V = X @ Wq, X @ Wk, X @ Wv\n    S = Q @ K.T / np.sqrt(d)\n    att = np.exp(S - S.max(1, keepdims=True))\n    att /= att.sum(1, keepdims=True)                 # 全局：每个节点看所有节点\n    X_T = X + att @ V\n    H = X_M + X_T                                    # 两条路相加\n    return H + np.maximum(H @ W1, 0) @ W2            # 再过一个 MLP\n\ndef embed(A, use_rwse):\n    feats = rwse(A) if use_rwse else np.ones((len(A), 4))\n    return gps_layer(feats @ W_in, A).sum(0)        # sum readout\n\nfor use in (False, True):\n    same = np.allclose(embed(hexagon, use), embed(two_triangles, use))\n    print(\"加上 RWSE\" if use else \"没有编码\", \"→ 两张图的表示相同:\", same)\n```\n\n```text\n没有编码 → 两张图的表示相同: True\n加上 RWSE → 两张图的表示相同: False\n```\n\n这个结果值得停一下。没有编码时，就算加了全局注意力，两张图还是分不开；所有节点的输入都一样，注意力也只能平均分配，看所有节点和看邻居没有差别。补上 RWSE，节点一开始就带着「自己在不在三角形上」的信息，同一层 GPS 就分得开了。\n\n### 线性的复杂度\n\n消息传递的成本是 $O(E)$。全局注意力如果用标准 Transformer 是 $O(N^2)$，换成 Performer 这类线性注意力就是 $O(N)$；因为 PE、SE 都放在节点与真正的边上，边特征又不进全局注意力，整个模型不需要建出完整的注意力矩阵。分子图、知识图谱这类稀疏图，边数和节点数成正比，整体就是 $O(N)$。前处理的拉普拉斯特征分解不算在内。\n\n实际的速度，在 ogbg-molpcba 上，一个约 600 万参数的模型每个 epoch 要 196 秒，同型 GPU 上的 SAN 要 883 秒。\n\n作者也证明，给定合适的组件，这个架构不会丢失边的信息，而且是图上的通用函数逼近器。\n\n## 讨论\n\n### 消融：两条路各自的份量\n\n作者在四个数据集上拆解了各个组件：\n\n| 设置 | ZINC MAE ↓ | PCQM4Mv2 子集 MAE ↓ | CIFAR10 准确率 ↑ | MalNet-Tiny 准确率 ↑ |\n| --- | ---: | ---: | ---: | ---: |\n| 完整的 GPS（Transformer） | 0.070 | 0.1159 | 72.31% | 93.50% |\n| 全局注意力换成 Performer | 0.071 | 0.1142 | 70.67% | 92.64% |\n| 全局注意力换成 BigBird | 0.071 | 0.1237 | 70.48% | 92.34% |\n| 拿掉全局注意力 | 0.070 | 0.1213 | 69.95% | 92.23% |\n| 拿掉消息传递 | 0.217 | 0.3294 | 68.86% | 73.90% |\n\n拿掉消息传递，四个数据集都大幅退步，MalNet-Tiny 从 93.50% 掉到 73.90%。作者认为消息传递是不可或缺的，因为没有它，边特征就完全没地方进来。\n\n全局注意力的贡献比较看数据集。ZINC 上拿不拿掉都是 0.070，作者解释，ZINC 要预测的分子性质只取决于局部子结构的数量，不需要全局连接，但很需要结构编码；其他三个数据集，Transformer 都有帮助。Performer 落后标准 Transformer，但能扩展到大图；BigBird 在这个设置下没有显著增益，还比 Performer 慢。\n\n编码方面，ZINC 上不加任何编码的 MAE 是 0.113，加 RWSE 是 0.070，加 LapPE 是 0.116。RWSE 以相对低的成本带来一致的增益；SignNet 是单项表现最好的编码，但计算成本比较高。\n\n### 和其他模型比\n\n在 Benchmarking GNNs 的五个数据集上，GPS 在 ZINC 拿到 0.070 的 MAE，比 GCN 的 0.367、GIN 的 0.526、Graphormer 的 0.122 都低；CIFAR10 是 72.30%；MNIST 和 CLUSTER 都排第二，PATTERN 则落在第四，和前几名只差零点几个百分点。\n\n在 OGB 的图层级任务上，GPS 在 ogbg-molpcba、ogbg-ppa、ogbg-code2 都排进前三，作者也指出它胜过所有其他的 Graph Transformer。PCQM4Mv2 上，1,940 万参数的 GPS-medium 验证集 MAE 是 0.0858，比 4,830 万参数的 Graphormer（0.0864）略好。\n\n在专门测长距离依赖的 Long Range Graph Benchmark（LRGB）上，GPS 用 GatedGCN 加 Transformer 加 LapPE，五个数据集里有四个胜过所有基准。例如 Peptides-func 的平均精确率，GCN 是 0.5930，GPS 是 0.6535；PascalVOC-SP 的 F1，GCN 是 0.1268，GPS 是 0.3748。\n\n### 一年后的重新评估\n\nLRGB 上的差距，后来被重新检验过。2023 年 9 月，RWTH Aachen 的 Tönshoff 等人替 GCN 等基准做了基本的超参数搜索，Peptides-func 上 GCN 升到 0.6860，超过 GPS 重跑的 0.6534；他们发现最关键的超参数，是 LRGB 原本设置的预测头只有一层线性层，换成两层 MLP 就补上了大部分的差距，GPS 换不换则差不多。[Tönshoff et al.](https://arxiv.org/abs/2309.00367v2)\n\n2025 年，Luo 等人把残差、归一化、FFN、位置编码等六个技巧加到 GCN、GIN、GatedGCN 上，在 14 个图层级数据集上和 GPS 等 Graph Transformer 比，结果打平甚至胜出。\n\n回头看，GPS 配方里的几样材料，位置与结构编码、残差、前馈层，后来也都出现在让经典 GNN 追上来的技巧清单里。\n\n\n> [!TIP]\n> \n> GPS 想解决的 over-squashing、LRGB 的五个数据集，以及这两篇重新评估的细节，可以参考我们的笔记：\n> \n> - **[[20.06] Over-squashing：塞不下了](/blog/papers/graph-learning/over-squashing)**\n> - **[[22.06] LRGB：远方的考题](/blog/papers/graph-learning/lrgb)**\n> - **[[23.09] Where Did the Gap Go?：差距去哪了？](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[25.02] GNN+：老骥伏枥](/blog/papers/graph-learning/gnn-plus)**\n\n\n## 结论\n\nGraphGPS 把 Graph Transformer 拆成三份材料，位置与结构编码、局部的消息传递、全局的注意力，每一份都可以替换。拿掉消息传递，模型全面退步；拿掉全局注意力，退多少要看任务需不需要远距离的信息；没有编码，加了注意力也看不见消息传递看不见的结构。\n\n如果要从这篇带走一件事，大概是 RWSE。它只需要几次稀疏矩阵乘法，加在任何 GNN 的输入上都很便宜。\n\n至少在 ZINC 上，拿掉全局注意力，MAE 一点都没变；拿掉 RWSE，MAE 从 0.070 退回 0.113。几步随机游走的回家概率，比一整层全局注意力还管用。\n","imageSizes":{"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png":{"width":1116,"height":798}}}