{"notes":[{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"圖上的 GPS","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/graphgps.9db287ae3c36.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"位置與結構","text":"位置與結構","depth":3},{"id":"為什麼訊息傳遞需要它們","text":"為什麼訊息傳遞需要它們","depth":3},{"id":"先傳遞再注意會先丟掉資訊","text":"先傳遞再注意，會先丟掉資訊","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"三份材料","text":"三份材料","depth":3},{"id":"一層-gps","text":"一層 GPS","depth":3},{"id":"線性的複雜度","text":"線性的複雜度","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"消融兩條路各自的份量","text":"消融：兩條路各自的份量","depth":3},{"id":"和其他模型比","text":"和其他模型比","depth":3},{"id":"一年後的重新評估","text":"一年後的重新評估","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/graph-transformer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graph Transformer","subtitle":"連滿反而糟","author":"Z. Yuan","paperDate":"2020-12-17","publishedOn":"2024-03-13","paperTitle":"A Generalization of Transformer Networks to Graphs","paperUrl":"https://arxiv.org/abs/2012.09699","bibliography":{"title":"A Generalization of Transformer Networks to Graphs","authors":["Vijay Prakash Dwivedi","Xavier Bresson"],"url":"https://arxiv.org/abs/2012.09699","pdfUrl":"https://arxiv.org/pdf/2012.09699v2","codeUrl":"https://github.com/graphdeeplearning/graphtransformer","preprint":{"id":"2012.09699","submittedOn":"2020-12-17","url":"https://arxiv.org/abs/2012.09699"},"publication":{"venue":"AAAI Workshop on Deep Learning on Graphs","shortVenue":"DLG-AAAI","year":2021,"kind":"workshop","url":"https://arxiv.org/abs/2012.09699v2"},"citation":"@misc{graphlearninggraphtransformer2020,\n  title = {{A Generalization of Transformer Networks to Graphs}},\n  author = {Vijay Prakash Dwivedi and Xavier Bresson},\n  year = {2020},\n  eprint = {2012.09699},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2012.09699}\n}\n","citationPath":"/content/papers/citations/graph-learning/graph-transformer.4e2f841bbbf0.bib"},"thumbnail":{"src":"/content/papers/assets/0540122812c6e6914f22240411cf2d172a94764bf8a0e6f6ab2cdae7c618f9fb.png","width":1324,"height":1000},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/graph-transformer.1a6743ef39d8.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"只在鄰居之間算注意力","text":"只在鄰居之間算注意力","depth":3},{"id":"拉普拉斯位置編碼","text":"拉普拉斯位置編碼","depth":3},{"id":"邊特徵與正規化","text":"邊特徵與正規化","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"實驗設定","text":"實驗設定","depth":3},{"id":"全連接會怎樣","text":"全連接會怎樣？","depth":3},{"id":"位置編碼與-batchnorm","text":"位置編碼與 BatchNorm","depth":3},{"id":"和-gnn-比","text":"和 GNN 比","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""}]},{"id":"graph-learning/san","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"SAN","subtitle":"聽見圖的形狀","author":"Z. Yuan","paperDate":"2021-06-07","publishedOn":"2024-03-22","world":"external/san","paperTitle":"Rethinking Graph Transformers with Spectral Attention","paperUrl":"https://arxiv.org/abs/2106.03893","bibliography":{"title":"Rethinking Graph Transformers with Spectral Attention","authors":["Devin Kreuzer","Dominique Beaini","William L. Hamilton","Vincent Létourneau","Prudencio Tossou"],"url":"https://arxiv.org/abs/2106.03893","pdfUrl":"https://arxiv.org/pdf/2106.03893v3","codeUrl":"https://github.com/DevinKreuzer/SAN","preprint":{"id":"2106.03893","submittedOn":"2021-06-07","url":"https://arxiv.org/abs/2106.03893"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/b4fd1d2cb085390fbbadae65e07876a7-Abstract.html"},"citation":"@misc{graphlearningsan2021,\n  title = {{Rethinking Graph Transformers with Spectral Attention}},\n  author = {Devin Kreuzer and Dominique Beaini and William L. Hamilton and Vincent Létourneau and Prudencio Tossou},\n  year = {2021},\n  eprint = {2106.03893},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.03893}\n}\n","citationPath":"/content/papers/citations/graph-learning/san.e69e469c3b91.bib"},"thumbnail":{"src":"/content/papers/assets/239f3ac016f803fed4fedec5cad345c48bc6b97791249f0e5cfdf2efd433b5c2.png","width":1280,"height":840},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/san.e84a43ca7dce.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"學出來的位置編碼","text":"學出來的位置編碼","depth":3},{"id":"全連接但分開兩套注意力","text":"全連接，但分開兩套注意力","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"gamma-該設多大","text":"\\gamma 該設多大？","depth":3},{"id":"全連接和-lpe-各自的貢獻","text":"全連接和 LPE 各自的貢獻","depth":3},{"id":"和當時最好的模型比","text":"和當時最好的模型比","depth":3},{"id":"代價","text":"代價","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""}]},{"id":"graph-learning/graphormer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graphormer","subtitle":"其實不差","author":"Z. Yuan","paperDate":"2021-06-09","publishedOn":"2024-03-28","world":"external/graphormer","paperTitle":"Do Transformers Really Perform Bad for Graph Representation?","paperUrl":"https://arxiv.org/abs/2106.05234","bibliography":{"title":"Do Transformers Really Perform Bad for Graph Representation?","authors":["Chengxuan Ying","Tianle Cai","Shengjie Luo","Shuxin Zheng","Guolin Ke","Di He","Yanming Shen","Tie-Yan Liu"],"url":"https://arxiv.org/abs/2106.05234","pdfUrl":"https://arxiv.org/pdf/2106.05234v5","codeUrl":"https://github.com/Microsoft/Graphormer","preprint":{"id":"2106.05234","submittedOn":"2021-06-09","url":"https://arxiv.org/abs/2106.05234"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/f1c1592588411002af340cbaedd6fc33-Abstract.html"},"citation":"@misc{graphlearninggraphormer2021,\n  title = {{Do Transformers Really Perform Bad for Graph Representation?}},\n  author = {Chengxuan Ying and Tianle Cai and Shengjie Luo and Shuxin Zheng and Guolin Ke and Di He and Yanming Shen and Tie-Yan Liu},\n  year = {2021},\n  eprint = {2106.05234},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.05234}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphormer.23b94dbe8ef3.bib"},"thumbnail":{"src":"/content/papers/assets/c2a2e0893f98e2ed6d93e1f4ee87040eeffbc63c4b955e6a319700c5e87f1847.png","width":872,"height":620},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/graphormer.848ae0c7733a.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"中心性編碼","text":"中心性編碼","depth":3},{"id":"空間編碼","text":"空間編碼","depth":3},{"id":"邊編碼","text":"邊編碼","depth":3},{"id":"其他細節","text":"其他細節","depth":3},{"id":"能表示哪些-gnn","text":"能表示哪些 GNN？","depth":3},{"id":"比-wl-更強","text":"比 WL 更強","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"pcqm4m-lsc","text":"PCQM4M-LSC","depth":3},{"id":"其他資料集","text":"其他資料集","depth":3},{"id":"每個編碼貢獻多少","text":"每個編碼貢獻多少","depth":3},{"id":"代價","text":"代價","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""}]}],"text":"2021 年，Graphormer 用 Transformer 拿下 OGB-LSC 量子化學競賽的冠軍，Graph Transformer 一下子熱了起來。\n\n道理不難理解。Transformer 讓每個節點直接看所有節點，訊息傳遞的幾個老問題，疊深會 over-smoothing、傳遠會 over-squashing、分辨能力卡在 WL 測試，好像都能一口氣繞過去。\n\n問題是，當時的 Graph Transformer 各做各的。有人用拉普拉斯的特徵向量當位置編碼，有人用最短路徑距離當注意力的偏差，有人把訊息傳遞層疊在前面；哪些設計真的有用、為什麼有用，沒有一個共同的說法。再加上全域注意力的成本是 $O(N^2)$，這些模型大多只能處理幾百個節點的小圖。\n\n\n> [!TIP]\n> \n> 只看鄰居、以拉普拉斯特徵向量當位置編碼的 GT，全連接但分開兩套注意力的 SAN，以及用最短路徑距離當注意力偏差的 Graphormer，可以參考我們的筆記：\n> \n> - **[[20.12] Graph Transformer：連滿反而糟](/blog/papers/graph-learning/graph-transformer)**\n> - **[[21.06] SAN：聽見圖的形狀](/blog/papers/graph-learning/san)**\n> - **[[21.06] Graphormer：其實不差](/blog/papers/graph-learning/graphormer)**\n\n\nMila 的 Ladislav Rampášek、Mikhail Galkin 與南洋理工大學的 Vijay Prakash Dwivedi 等人想把這些整理成一份配方。標題裡的 GPS，是 General、Powerful、Scalable 三個字的縮寫。\n\n## 定義問題\n\n### 位置與結構\n\nTransformer 本身看不到圖。把所有節點連在一起之後，誰和誰本來相鄰，模型完全不知道；所以 Graph Transformer 都得額外提供圖的資訊。\n\n作者先把這些額外資訊分成兩類：\n\n- **位置編碼（PE）**：告訴模型一個節點在圖上的「位置」。圖上相近的兩個節點，PE 也應該相近。常見的是拉普拉斯矩陣的特徵向量（LapPE）。\n- **結構編碼（SE）**：描述節點或整張圖的結構。兩個節點周圍的子圖長得像，SE 就應該相近。常見的是隨機遊走矩陣 $m$ 次方的對角線（RWSE），也就是從自己出發、走 $m$ 步回到自己的機率。\n\n兩類各自再分成局部、全域與相對三種，例如全域 SE 可以是拉普拉斯矩陣的特徵值，相對 SE 可以是兩個節點是否位在同一個子結構裡。作者把當時的做法整理成一張表，並主張 PE 與 SE 最好以節點特徵或真正的邊來表示，才能搭配線性複雜度的注意力。\n\n### 為什麼訊息傳遞需要它們\n\n標準的訊息傳遞網路，分辨能力和 1-WL 測試一樣。作者用一組 Circular Skip Link 圖說明這件事。兩張不同構的圖，WL 對每個節點都塗上同一個顏色，任何訊息傳遞網路都分不出來；加上全域 PE，每個節點一開始就有獨特的顏色，兩張圖就分得開；換成局部的 RWSE，也能抓到兩張圖跳接方式的差異。\n\n換句話說，這些資訊無法單靠訊息傳遞從圖結構裡學出來，得另外給。\n\n拿一個更小的例子，六個節點的六邊形，和兩個分開的三角形。兩張圖裡每個節點都有兩個鄰居，WL 分不開。算一下 RWSE：\n\n```python\nimport numpy as np\nnp.set_printoptions(precision=3, suppress=True)\n\ndef graph(edges, n=6):\n    A = np.zeros((n, n))\n    for i, j in edges:\n        A[i, j] = A[j, i] = 1\n    return A\n\nhexagon = graph([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5), (5, 0)])\ntwo_triangles = graph([(0, 1), (1, 2), (2, 0), (3, 4), (4, 5), (5, 3)])\n\ndef rwse(A, steps=4):\n    P = A / A.sum(1, keepdims=True)                  # 隨機遊走的轉移矩陣 D^-1 A\n    return np.stack([np.diag(np.linalg.matrix_power(P, m)) for m in range(1, steps + 1)], axis=1)\n\nprint(\"六邊形      0 號節點:\", rwse(hexagon)[0])\nprint(\"兩個三角形  0 號節點:\", rwse(two_triangles)[0])\n```\n\n```text\n六邊形      0 號節點: [0.    0.5   0.    0.375]\n兩個三角形  0 號節點: [0.    0.5   0.25  0.375]\n```\n\n差別在第三個數字，走 3 步回到自己的機率。三角形上有 0.25，六邊形上是 0，因為六邊形沒有長度為 3 的環。作者也提到，$m$ 為奇數時，RWSE 可以看出節點是否在長度為 $m$ 的環上。\n\n### 先傳遞再注意，會先丟掉資訊\n\n另一個常見的架構，是先疊幾層訊息傳遞，再接 Transformer，例如 GraphTrans。作者認為這樣有風險。前面的訊息傳遞層本身就有 over-smoothing、over-squashing 和表達力的限制，可能在一開始就把某些資訊丟掉，後面的 Transformer 再強也救不回來。\n\n電腦視覺裡「先卷積、再 Transformer」能成功，是因為網格上的卷積層表達力很強；作者不認為這套做法在圖上能同樣成立。\n\n## 解決問題\n\n### 三份材料\n\nGPS 的配方分三部分：\n\n1. **位置／結構編碼**：在輸入時加到節點特徵，或邊特徵上。\n2. **局部的訊息傳遞**：只沿著真正的邊，可以處理邊特徵。\n3. **全域的注意力**：讓每個節點看所有節點。\n\n每一部分都可以替換。訊息傳遞可以用 GINE、GatedGCN 或 PNA，全域注意力可以用標準的 Transformer、Performer 或 BigBird。\n\n[![原論文 Figure 1：模組化的 GPS 架構。左側三欄分別是位置編碼、結構編碼與圖特徵，各自經過 MLP、DeepSet 或 SignNet 等處理後加到節點與邊特徵上；右側是重複 L 次的 GPS 層，每層同時走「任一訊息傳遞層（GINE、GatedGCN、PNA）」與「任一全域注意力（Transformer、Performer、BigBird）」，兩者相加後接一個兩層 MLP。](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)\n\n*原論文 Figure 1（Rampášek 等人），取自 [**PDF 第 3 頁**](https://arxiv.org/pdf/2205.12454v4#page=3)。圖說未收錄。*\n\n### 一層 GPS\n\n每一層，兩條路並行，結果相加後再過一個 MLP：\n\n$$\nX_M^{\\ell+1},\\,E^{\\ell+1}=\\mathrm{MPNN}_e^{\\ell}\\left(X^\\ell,E^\\ell,A\\right)\n$$\n\n$$\nX_T^{\\ell+1}=\\mathrm{GlobalAttn}^{\\ell}\\left(X^\\ell\\right)\n$$\n\n$$\nX^{\\ell+1}=\\mathrm{MLP}^{\\ell}\\left(X_M^{\\ell+1}+X_T^{\\ell+1}\\right)\n$$\n\n邊特徵只進訊息傳遞那一條。式子裡省略了殘差連接和 BatchNorm，實際的實作兩條路都有。\n\n接著上面的兩張圖，用隨機權重寫一層 GPS。訊息傳遞用最簡單的鄰居總和，全域注意力用單頭的 Transformer 注意力，最後把所有節點加總，當成整張圖的表示：\n\n```python\nrng = np.random.default_rng(0)\nd = 4\nW_in = rng.normal(size=(4, d))                       # 把輸入特徵投影到 d 維\nWm, Wq, Wk, Wv = (rng.normal(size=(d, d)) for _ in range(4))\nW1, W2 = rng.normal(size=(d, 2 * d)), rng.normal(size=(2 * d, d))\n\ndef gps_layer(X, A):\n    X_M = X + np.maximum((A @ X) @ Wm, 0)            # 局部：鄰居總和的訊息傳遞，加殘差\n    Q, K, V = X @ Wq, X @ Wk, X @ Wv\n    S = Q @ K.T / np.sqrt(d)\n    att = np.exp(S - S.max(1, keepdims=True))\n    att /= att.sum(1, keepdims=True)                 # 全域：每個節點看所有節點\n    X_T = X + att @ V\n    H = X_M + X_T                                    # 兩條路相加\n    return H + np.maximum(H @ W1, 0) @ W2            # 再過一個 MLP\n\ndef embed(A, use_rwse):\n    feats = rwse(A) if use_rwse else np.ones((len(A), 4))\n    return gps_layer(feats @ W_in, A).sum(0)        # sum readout\n\nfor use in (False, True):\n    same = np.allclose(embed(hexagon, use), embed(two_triangles, use))\n    print(\"加上 RWSE\" if use else \"沒有編碼\", \"→ 兩張圖的表示相同:\", same)\n```\n\n```text\n沒有編碼 → 兩張圖的表示相同: True\n加上 RWSE → 兩張圖的表示相同: False\n```\n\n這個結果值得停一下。沒有編碼時，就算加了全域注意力，兩張圖還是分不開；所有節點的輸入都一樣，注意力也只能平均分配，看所有節點和看鄰居沒有差別。補上 RWSE，節點一開始就帶著「自己在不在三角形上」的資訊，同一層 GPS 就分得開了。\n\n### 線性的複雜度\n\n訊息傳遞的成本是 $O(E)$。全域注意力如果用標準 Transformer 是 $O(N^2)$，換成 Performer 這類線性注意力就是 $O(N)$；因為 PE、SE 都放在節點與真正的邊上，邊特徵又不進全域注意力，整個模型不需要建出完整的注意力矩陣。分子圖、知識圖譜這類稀疏圖，邊數和節點數成正比，整體就是 $O(N)$。前處理的拉普拉斯特徵分解不算在內。\n\n實際的速度，在 ogbg-molpcba 上，一個約 600 萬參數的模型每個 epoch 要 196 秒，同型 GPU 上的 SAN 要 883 秒。\n\n作者也證明，給定合適的元件，這個架構不會丟失邊的資訊，而且是圖上的通用函數逼近器。\n\n## 討論\n\n### 消融：兩條路各自的份量\n\n作者在四個資料集上拆解了各個元件：\n\n| 設定 | ZINC MAE ↓ | PCQM4Mv2 子集 MAE ↓ | CIFAR10 準確率 ↑ | MalNet-Tiny 準確率 ↑ |\n| --- | ---: | ---: | ---: | ---: |\n| 完整的 GPS（Transformer） | 0.070 | 0.1159 | 72.31% | 93.50% |\n| 全域注意力換成 Performer | 0.071 | 0.1142 | 70.67% | 92.64% |\n| 全域注意力換成 BigBird | 0.071 | 0.1237 | 70.48% | 92.34% |\n| 拿掉全域注意力 | 0.070 | 0.1213 | 69.95% | 92.23% |\n| 拿掉訊息傳遞 | 0.217 | 0.3294 | 68.86% | 73.90% |\n\n拿掉訊息傳遞，四個資料集都大幅退步，MalNet-Tiny 從 93.50% 掉到 73.90%。作者認為訊息傳遞是不可或缺的，因為沒有它，邊特徵就完全沒地方進來。\n\n全域注意力的貢獻比較看資料集。ZINC 上拿不拿掉都是 0.070，作者解釋，ZINC 要預測的分子性質只取決於局部子結構的數量，不需要全域連線，但很需要結構編碼；其他三個資料集，Transformer 都有幫助。Performer 落後標準 Transformer，但能擴展到大圖；BigBird 在這個設定下沒有顯著增益，還比 Performer 慢。\n\n編碼方面，ZINC 上不加任何編碼的 MAE 是 0.113，加 RWSE 是 0.070，加 LapPE 是 0.116。RWSE 以相對低的成本帶來一致的增益；SignNet 是單項表現最好的編碼，但計算成本比較高。\n\n### 和其他模型比\n\n在 Benchmarking GNNs 的五個資料集上，GPS 在 ZINC 拿到 0.070 的 MAE，比 GCN 的 0.367、GIN 的 0.526、Graphormer 的 0.122 都低；CIFAR10 是 72.30%；MNIST 和 CLUSTER 都排第二，PATTERN 則落在第四，和前幾名只差零點幾個百分點。\n\n在 OGB 的圖層級任務上，GPS 在 ogbg-molpcba、ogbg-ppa、ogbg-code2 都排進前三，作者也指出它勝過所有其他的 Graph Transformer。PCQM4Mv2 上，1,940 萬參數的 GPS-medium 驗證集 MAE 是 0.0858，比 4,830 萬參數的 Graphormer（0.0864）略好。\n\n在專門測長距離依賴的 Long Range Graph Benchmark（LRGB）上，GPS 用 GatedGCN 加 Transformer 加 LapPE，五個資料集裡有四個勝過所有基準。例如 Peptides-func 的平均精確率，GCN 是 0.5930，GPS 是 0.6535；PascalVOC-SP 的 F1，GCN 是 0.1268，GPS 是 0.3748。\n\n### 一年後的重新評估\n\nLRGB 上的差距，後來被重新檢驗過。2023 年 9 月，RWTH Aachen 的 Tönshoff 等人替 GCN 等基準做了基本的超參數搜尋，Peptides-func 上 GCN 升到 0.6860，超過 GPS 重跑的 0.6534；他們發現最關鍵的超參數，是 LRGB 原本設定的預測頭只有一層線性層，換成兩層 MLP 就補上了大部分的差距，GPS 換不換則差不多。[Tönshoff et al.](https://arxiv.org/abs/2309.00367v2)\n\n2025 年，Luo 等人把殘差、正規化、FFN、位置編碼等六個技巧加到 GCN、GIN、GatedGCN 上，在 14 個圖層級資料集上和 GPS 等 Graph Transformer 比，結果打平甚至勝出。\n\n回頭看，GPS 配方裡的幾樣材料，位置與結構編碼、殘差、前饋層，後來也都出現在讓經典 GNN 追上來的技巧清單裡。\n\n\n> [!TIP]\n> \n> GPS 想解決的 over-squashing、LRGB 的五個資料集，以及這兩篇重新評估的細節，可以參考我們的筆記：\n> \n> - **[[20.06] Over-squashing：塞不下了](/blog/papers/graph-learning/over-squashing)**\n> - **[[22.06] LRGB：遠方的考題](/blog/papers/graph-learning/lrgb)**\n> - **[[23.09] Where Did the Gap Go?：差距去哪了？](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[25.02] GNN+：老驥伏櫪](/blog/papers/graph-learning/gnn-plus)**\n\n\n## 結論\n\nGraphGPS 把 Graph Transformer 拆成三份材料，位置與結構編碼、局部的訊息傳遞、全域的注意力，每一份都可以替換。拿掉訊息傳遞，模型全面退步；拿掉全域注意力，退多少要看任務需不需要遠距離的資訊；沒有編碼，加了注意力也看不見訊息傳遞看不見的結構。\n\n如果要從這篇帶走一件事，大概是 RWSE。它只需要幾次稀疏矩陣乘法，加在任何 GNN 的輸入上都很便宜。\n\n至少在 ZINC 上，拿掉全域注意力，MAE 一點都沒變；拿掉 RWSE，MAE 從 0.070 退回 0.113。幾步隨機遊走的回家機率，比一整層全域注意力還管用。\n","imageSizes":{"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png":{"width":1116,"height":798}}}