{"notes":[{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"グラフ上の GPS","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"ja","path":"/content/papers/ja/graph-learning/graphgps.0d5dc280bee1.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"位置と構造","text":"位置と構造","depth":3},{"id":"なぜ伝播にも必要か","text":"なぜ伝播にも必要か","depth":3},{"id":"伝播のあとで注意しても遅い","text":"伝播のあとで注意しても遅い","depth":3},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの材料","text":"三つの材料","depth":3},{"id":"gps-の一層","text":"GPS の一層","depth":3},{"id":"線形の計算量","text":"線形の計算量","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"アブレーション二経路の寄与","text":"アブレーション：二経路の寄与","depth":3},{"id":"他モデルとの比較","text":"他モデルとの比較","depth":3},{"id":"一年後の再評価","text":"一年後の再評価","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/graph-transformer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graph Transformer","subtitle":"全部つなぐと、かえって悪い","author":"Z. Yuan","paperDate":"2020-12-17","publishedOn":"2024-03-13","paperTitle":"A Generalization of Transformer Networks to Graphs","paperUrl":"https://arxiv.org/abs/2012.09699","bibliography":{"title":"A Generalization of Transformer Networks to Graphs","authors":["Vijay Prakash Dwivedi","Xavier Bresson"],"url":"https://arxiv.org/abs/2012.09699","pdfUrl":"https://arxiv.org/pdf/2012.09699v2","codeUrl":"https://github.com/graphdeeplearning/graphtransformer","preprint":{"id":"2012.09699","submittedOn":"2020-12-17","url":"https://arxiv.org/abs/2012.09699"},"publication":{"venue":"AAAI Workshop on Deep Learning on Graphs","shortVenue":"DLG-AAAI","year":2021,"kind":"workshop","url":"https://arxiv.org/abs/2012.09699v2"},"citation":"@misc{graphlearninggraphtransformer2020,\n  title = {{A Generalization of Transformer Networks to Graphs}},\n  author = {Vijay Prakash Dwivedi and Xavier Bresson},\n  year = {2020},\n  eprint = {2012.09699},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2012.09699}\n}\n","citationPath":"/content/papers/citations/graph-learning/graph-transformer.4e2f841bbbf0.bib"},"thumbnail":{"src":"/content/papers/assets/0540122812c6e6914f22240411cf2d172a94764bf8a0e6f6ab2cdae7c618f9fb.png","width":1324,"height":1000},"locale":"ja","path":"/content/papers/ja/graph-learning/graph-transformer.41cd0606ebf1.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"隣接ノード間だけの注意","text":"隣接ノード間だけの注意","depth":3},{"id":"ラプラシアン位置符号化","text":"ラプラシアン位置符号化","depth":3},{"id":"エッジ特徴量と正規化","text":"エッジ特徴量と正規化","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"実験設定","text":"実験設定","depth":3},{"id":"全結合にすると","text":"全結合にすると？","depth":3},{"id":"位置符号化と-batchnorm","text":"位置符号化と BatchNorm","depth":3},{"id":"gnn-との比較","text":"GNN との比較","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""}]},{"id":"graph-learning/san","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"SAN","subtitle":"グラフの形を聴く","author":"Z. Yuan","paperDate":"2021-06-07","publishedOn":"2024-03-22","world":"external/san","paperTitle":"Rethinking Graph Transformers with Spectral Attention","paperUrl":"https://arxiv.org/abs/2106.03893","bibliography":{"title":"Rethinking Graph Transformers with Spectral Attention","authors":["Devin Kreuzer","Dominique Beaini","William L. Hamilton","Vincent Létourneau","Prudencio Tossou"],"url":"https://arxiv.org/abs/2106.03893","pdfUrl":"https://arxiv.org/pdf/2106.03893v3","codeUrl":"https://github.com/DevinKreuzer/SAN","preprint":{"id":"2106.03893","submittedOn":"2021-06-07","url":"https://arxiv.org/abs/2106.03893"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/b4fd1d2cb085390fbbadae65e07876a7-Abstract.html"},"citation":"@misc{graphlearningsan2021,\n  title = {{Rethinking Graph Transformers with Spectral Attention}},\n  author = {Devin Kreuzer and Dominique Beaini and William L. Hamilton and Vincent Létourneau and Prudencio Tossou},\n  year = {2021},\n  eprint = {2106.03893},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.03893}\n}\n","citationPath":"/content/papers/citations/graph-learning/san.e69e469c3b91.bib"},"thumbnail":{"src":"/content/papers/assets/239f3ac016f803fed4fedec5cad345c48bc6b97791249f0e5cfdf2efd433b5c2.png","width":1280,"height":840},"locale":"ja","path":"/content/papers/ja/graph-learning/san.e02e8dcf7c2d.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"学習する位置符号化","text":"学習する位置符号化","depth":3},{"id":"全結合に二系統の注意","text":"全結合に二系統の注意","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"gamma-はどれだけ","text":"\\gamma はどれだけ？","depth":3},{"id":"全結合と-lpe-の寄与","text":"全結合と LPE の寄与","depth":3},{"id":"当時の最良と比べる","text":"当時の最良と比べる","depth":3},{"id":"代償","text":"代償","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""}]},{"id":"graph-learning/graphormer","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Graphormer","subtitle":"実は悪くない","author":"Z. Yuan","paperDate":"2021-06-09","publishedOn":"2024-03-28","world":"external/graphormer","paperTitle":"Do Transformers Really Perform Bad for Graph Representation?","paperUrl":"https://arxiv.org/abs/2106.05234","bibliography":{"title":"Do Transformers Really Perform Bad for Graph Representation?","authors":["Chengxuan Ying","Tianle Cai","Shengjie Luo","Shuxin Zheng","Guolin Ke","Di He","Yanming Shen","Tie-Yan Liu"],"url":"https://arxiv.org/abs/2106.05234","pdfUrl":"https://arxiv.org/pdf/2106.05234v5","codeUrl":"https://github.com/Microsoft/Graphormer","preprint":{"id":"2106.05234","submittedOn":"2021-06-09","url":"https://arxiv.org/abs/2106.05234"},"publication":{"venue":"NeurIPS","year":2021,"kind":"conference","url":"https://proceedings.neurips.cc/paper/2021/hash/f1c1592588411002af340cbaedd6fc33-Abstract.html"},"citation":"@misc{graphlearninggraphormer2021,\n  title = {{Do Transformers Really Perform Bad for Graph Representation?}},\n  author = {Chengxuan Ying and Tianle Cai and Shengjie Luo and Shuxin Zheng and Guolin Ke and Di He and Yanming Shen and Tie-Yan Liu},\n  year = {2021},\n  eprint = {2106.05234},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2106.05234}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphormer.23b94dbe8ef3.bib"},"thumbnail":{"src":"/content/papers/assets/c2a2e0893f98e2ed6d93e1f4ee87040eeffbc63c4b955e6a319700c5e87f1847.png","width":872,"height":620},"locale":"ja","path":"/content/papers/ja/graph-learning/graphormer.4902f54c82d1.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"中心性符号化","text":"中心性符号化","depth":3},{"id":"空間符号化","text":"空間符号化","depth":3},{"id":"エッジ符号化","text":"エッジ符号化","depth":3},{"id":"ほかの細部","text":"ほかの細部","depth":3},{"id":"どの-gnn-を表せるか","text":"どの GNN を表せるか","depth":3},{"id":"wl-より強い","text":"WL より強い","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"pcqm4m-lsc","text":"PCQM4M-LSC","depth":3},{"id":"ほかのデータセット","text":"ほかのデータセット","depth":3},{"id":"各符号化の寄与","text":"各符号化の寄与","depth":3},{"id":"代償","text":"代償","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""}]}],"text":"2021 年、Graphormer が OGB-LSC の量子化学コンペで優勝し、Graph Transformer が一気に注目されました。\n\n全ノードを直接見れば、深さの over-smoothing、距離の over-squashing、WL の識別上限を一度に迂回できそうです。\n\nしかし当時は、ラプラシアン位置符号化、最短距離の注意バイアス、前段の伝播などが個別に使われ、何がなぜ効くかの共通説明がありませんでした。$O(N^2)$ の大域的注意のため、多くは数百ノードだけを扱います。\n\n\n> [!TIP]\n> \n> 隣だけを見る GT、二系統の注意の SAN、最短距離バイアスの Graphormer は、こちらのノートをどうぞ。\n> \n> - **[[20.12] Graph Transformer：全部つなぐと、かえって悪い](/blog/papers/graph-learning/graph-transformer)**\n> - **[[21.06] SAN：グラフの形を聴く](/blog/papers/graph-learning/san)**\n> - **[[21.06] Graphormer：実は悪くない](/blog/papers/graph-learning/graphormer)**\n\n\nMila の Ladislav Rampášek、Mikhail Galkin、南洋理工大学の Vijay Prakash Dwivedi らは、一つのレシピに整理します。GPS は General、Powerful、Scalable の略です。\n\n## 問題設定\n\n### 位置と構造\n\nTransformer 自体にはグラフが見えません。全結合後は元の隣接関係を知らないので、追加情報が必要です。\n\n二種類に分けます。\n\n- **位置符号化（PE）**：グラフ内の位置を伝え、近いノードは近い PE を持つべきです。ラプラシアン固有ベクトル（LapPE）が代表です。\n- **構造符号化（SE）**：ノード周辺や全グラフの構造です。似た部分グラフなら近い SE を持つべきです。ランダムウォーク行列の $m$ 乗の対角（RWSE）、すなわち $m$ 歩の帰還確率が代表です。\n\nそれぞれ局所・大域・相対に分けます。大域 SE は固有値、相対 SE は同じ部分構造にいるか、などです。既存手法を表にし、線形注意と組み合わせるため、PE、SE をノード特徴か実エッジで表すことを勧めます。\n\n### なぜ伝播にも必要か\n\n標準伝播の識別力は 1-WL と同じです。Circular Skip Link の非同型二グラフでも WL は全ノード同色にし、伝播では区別できません。大域 PE で固有の色を与えれば分けられ、局所 RWSE でもスキップ接続の差を捉えます。\n\nこうした情報は伝播だけでは構造から学べず、別途与える必要があります。\n\n六角形と、互いに離れた二つの三角形で見ます。各ノードの隣は二つで WL は区別できません。RWSE を計算します。\n\n```python\nimport numpy as np\nnp.set_printoptions(precision=3, suppress=True)\n\ndef graph(edges, n=6):\n    A = np.zeros((n, n))\n    for i, j in edges:\n        A[i, j] = A[j, i] = 1\n    return A\n\nhexagon = graph([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5), (5, 0)])\ntwo_triangles = graph([(0, 1), (1, 2), (2, 0), (3, 4), (4, 5), (5, 3)])\n\ndef rwse(A, steps=4):\n    P = A / A.sum(1, keepdims=True)                  # ランダムウォークの遷移行列 D^-1 A\n    return np.stack([np.diag(np.linalg.matrix_power(P, m)) for m in range(1, steps + 1)], axis=1)\n\nprint(\"六角形      ノード 0:\", rwse(hexagon)[0])\nprint(\"二つの三角形  ノード 0:\", rwse(two_triangles)[0])\n```\n\n```text\n六角形      ノード 0: [0.    0.5   0.    0.375]\n二つの三角形  ノード 0: [0.    0.5   0.25  0.375]\n```\n\n差は三つ目、三歩の帰還確率です。三角形は 0.25、六角形は長さ 3 の環がなく 0 です。奇数 $m$ の RWSE で長さ $m$ の環上にいるか分かるとも述べます。\n\n### 伝播のあとで注意しても遅い\n\nGraphTrans のように前段で数層伝播してから Transformer に渡す方法にはリスクがあります。伝播の over-smoothing、over-squashing、表現力の制約で先に情報を失うと、後段で取り戻せません。\n\nビジョンの「畳み込みのあと Transformer」は格子畳み込みが強いから成功し、同じことがグラフでも成り立つとは考えていません。\n\n## 解決方法\n\n### 三つの材料\n\nレシピは三つです。\n\n1. **位置・構造符号化**：入力でノードかエッジに加えます。\n2. **局所メッセージパッシング**：実エッジだけに沿い、エッジ特徴量も扱えます。\n3. **大域的注意**：全ノードが全ノードを見ます。\n\nすべて交換可能です。伝播は GINE、GatedGCN、PNA、注意は Transformer、Performer、BigBird を使えます。\n\n[![原論文 Figure 1：左の位置符号化、構造符号化、グラフ特徴量を MLP、DeepSet、SignNet などに通し、ノード・エッジに加える。右は L 回繰り返す GPS 層で、任意の伝播（GINE、GatedGCN、PNA）と任意の大域注意（Transformer、Performer、BigBird）を並行し、足して二層 MLP に通す。](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)](/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png)\n\n*原論文 Figure 1（Rampášek ら）、[**PDF 3 ページ**](https://arxiv.org/pdf/2205.12454v4#page=3)より。原キャプションは省略。*\n\n### GPS の一層\n\n二経路を並行し、結果を足して MLP に通します。\n\n$$\nX_M^{\\ell+1},\\,E^{\\ell+1}=\\mathrm{MPNN}_e^{\\ell}\\left(X^\\ell,E^\\ell,A\\right)\n$$\n\n$$\nX_T^{\\ell+1}=\\mathrm{GlobalAttn}^{\\ell}\\left(X^\\ell\\right)\n$$\n\n$$\nX^{\\ell+1}=\\mathrm{MLP}^{\\ell}\\left(X_M^{\\ell+1}+X_T^{\\ell+1}\\right)\n$$\n\nエッジ特徴量は伝播側だけです。式で省略した残差と BatchNorm は、実装では両側にあります。\n\n先の二グラフでランダム重みの一層を作ります。伝播は単純な近傍総和、注意は一ヘッド Transformer で、最後に全ノードを足してグラフ表現にします。\n\n```python\nrng = np.random.default_rng(0)\nd = 4\nW_in = rng.normal(size=(4, d))                       # 入力特徴量を d 次元へ投影\nWm, Wq, Wk, Wv = (rng.normal(size=(d, d)) for _ in range(4))\nW1, W2 = rng.normal(size=(d, 2 * d)), rng.normal(size=(2 * d, d))\n\ndef gps_layer(X, A):\n    X_M = X + np.maximum((A @ X) @ Wm, 0)            # 局所：隣の総和による伝播と残差\n    Q, K, V = X @ Wq, X @ Wk, X @ Wv\n    S = Q @ K.T / np.sqrt(d)\n    att = np.exp(S - S.max(1, keepdims=True))\n    att /= att.sum(1, keepdims=True)                 # 大域：全ノードから全ノードを見る\n    X_T = X + att @ V\n    H = X_M + X_T                                    # 二経路を加算\n    return H + np.maximum(H @ W1, 0) @ W2            # さらに MLP を通す\n\ndef embed(A, use_rwse):\n    feats = rwse(A) if use_rwse else np.ones((len(A), 4))\n    return gps_layer(feats @ W_in, A).sum(0)        # sum readout\n\nfor use in (False, True):\n    same = np.allclose(embed(hexagon, use), embed(two_triangles, use))\n    print(\"RWSE あり\" if use else \"符号化なし\", \"→ 二グラフの表現が同じ:\", same)\n```\n\n```text\n符号化なし → 二グラフの表現が同じ: True\nRWSE あり → 二グラフの表現が同じ: False\n```\n\n符号化なしでは、大域的注意を加えても区別できません。全入力が同じなので注意は均等になり、全体を見るのも近傍を見るのも同じです。RWSE で三角形の有無を先に持たせると、同じ一層で分けられます。\n\n### 線形の計算量\n\n伝播は $O(E)$ です。標準注意は $O(N^2)$ ですが、Performer なら $O(N)$ です。PE、SE はノードと実エッジにあり、エッジ特徴量は大域注意に入らないので、完全な注意行列を作らずに済みます。エッジ数がノード数に比例する分子や知識グラフでは全体が $O(N)$ です。前処理の固有値分解は別です。\n\nogbg-molpcba の約 600 万パラメーターのモデルは一 epoch 196 秒で、同型 GPU の SAN は 883 秒です。\n\n適切な部品ならエッジ情報を失わず、グラフ上の汎用関数近似器になるとも証明します。\n\n## 議論\n\n### アブレーション：二経路の寄与\n\n四データセットで分解します。\n\n| 設定 | ZINC MAE ↓ | PCQM4Mv2 部分集合 MAE ↓ | CIFAR10 精度 ↑ | MalNet-Tiny 精度 ↑ |\n| --- | ---: | ---: | ---: | ---: |\n| 完全な GPS（Transformer） | 0.070 | 0.1159 | 72.31% | 93.50% |\n| 大域的注意を Performer に | 0.071 | 0.1142 | 70.67% | 92.64% |\n| 大域的注意を BigBird に | 0.071 | 0.1237 | 70.48% | 92.34% |\n| 大域的注意を除く | 0.070 | 0.1213 | 69.95% | 92.23% |\n| 伝播を除く | 0.217 | 0.3294 | 68.86% | 73.90% |\n\n伝播を除くと全て大幅悪化し、MalNet-Tiny は 93.50% から 73.90% です。エッジ特徴量の入口がなくなるので不可欠と考えます。\n\n注意の寄与はタスク次第です。ZINC は有無によらず 0.070。局所部分構造の個数だけで性質が決まり、大域接続は不要でも構造符号化は重要です。他三つでは有効です。Performer は標準より劣るが大規模へ広がり、BigBird は有意な利点なく Performer より遅くなります。\n\nZINC は符号化なし 0.113、RWSE 0.070、LapPE 0.116 です。RWSE は低コストで一貫して改善し、単体最良の SignNet は高コストです。\n\n### 他モデルとの比較\n\nBenchmarking GNNs 五つでは ZINC 0.070 で GCN 0.367、GIN 0.526、Graphormer 0.122 より低く、CIFAR10 は 72.30%。MNIST、CLUSTER は二位、PATTERN は四位で上位との差は 1 ポイント未満です。\n\nOGB の ogbg-molpcba、ogbg-ppa、ogbg-code2 で三位以内、他 Graph Transformer をすべて上回ると述べます。PCQM4Mv2 は 1,940 万 GPS-medium が検証 0.0858、4,830 万 Graphormer の 0.0864 より少しよくなります。\n\n長距離の LRGB は GatedGCN、Transformer、LapPE を使い、五つ中四つで全ベースラインに勝ちます。Peptides-func の平均適合率は GCN 0.5930、GPS 0.6535、PascalVOC-SP の F1 は GCN 0.1268、GPS 0.3748 です。\n\n### 一年後の再評価\n\n2023 年 9 月、RWTH Aachen の Tönshoff らが基本的な探索で GCN を Peptides-func 0.6860 に上げ、GPS 再実行の 0.6534 を超えました。鍵は一線形層だった予測ヘッドで、二層 MLP に替えると差の大半が埋まり、GPS はほぼ変わりませんでした。[Tönshoff et al.](https://arxiv.org/abs/2309.00367v2)\n\n2025 年、Luo らは残差、正規化、FFN、位置符号化など六技巧を GCN、GIN、GatedGCN に加え、14 のグラフレベルで GPS などと同等以上になりました。\n\nGPS の材料のうち、位置・構造符号化、残差、FFN は、古典 GNN が追いつく工夫にも入っています。\n\n\n> [!TIP]\n> \n> over-squashing、LRGB の五セット、二つの再評価は、こちらのノートをどうぞ。\n> \n> - **[[20.06] Over-squashing：もう入りきらない](/blog/papers/graph-learning/over-squashing)**\n> - **[[22.06] LRGB：遠くからの出題](/blog/papers/graph-learning/lrgb)**\n> - **[[23.09] Where Did the Gap Go?：差はどこへ消えた？](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[25.02] GNN+：老いた駿馬、志はいまだ遠く](/blog/papers/graph-learning/gnn-plus)**\n\n\n## 結論\n\n交換可能な三材料、位置・構造符号化、局所伝播、大域注意に分けました。伝播を除くと全面悪化し、注意の必要性は遠距離依存次第です。符号化がなければ注意を足しても、伝播に見えない構造は見えません。\n\n一つ持ち帰るなら RWSE でしょう。数回の疎行列積だけで、どの GNN の入力にも安く加えられます。\n\n少なくとも ZINC では、全注意を除いても MAE は同じですが、RWSE を除くと 0.070 から 0.113 に戻ります。数歩の帰還確率が、大域注意一層より効くのです。\n","imageSizes":{"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png":{"width":1116,"height":798}}}