{"notes":[{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"調整してから比べよう","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"ja","path":"/content/papers/ja/graph-learning/classic-gnns.2907699f1731.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの古典モデル","text":"三つの古典モデル","depth":3},{"id":"四つのハイパーパラメーター","text":"四つのハイパーパラメーター","depth":3},{"id":"実験設定","text":"実験設定","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"同類接続","text":"同類接続","depth":3},{"id":"異類接続","text":"異類接続","depth":3},{"id":"大規模グラフ","text":"大規模グラフ","depth":3},{"id":"何が鍵か","text":"何が鍵か","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/deeper-insights","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Deeper Insights","subtitle":"ならしすぎたグラフ","author":"Z. Yuan","paperDate":"2018-01-22","publishedOn":"2023-11-29","paperTitle":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","paperUrl":"https://arxiv.org/abs/1801.07606","bibliography":{"title":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","authors":["Qimai Li","Zhichao Han","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/1801.07606","pdfUrl":"https://arxiv.org/pdf/1801.07606v1","codeUrl":null,"preprint":{"id":"1801.07606","submittedOn":"2018-01-22","url":"https://arxiv.org/abs/1801.07606"},"publication":{"venue":"AAAI","year":2018,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/11604"},"citation":"@misc{graphlearningdeeperinsights2018,\n  title = {{Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning}},\n  author = {Qimai Li and Zhichao Han and Xiao-Ming Wu},\n  year = {2018},\n  eprint = {1801.07606},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.07606}\n}\n","citationPath":"/content/papers/citations/graph-learning/deeper-insights.52354c86fac9.bib"},"thumbnail":{"src":"/content/papers/assets/2a13880b48fae6456e19c622cee1d7c7907f8c31d6890722d4980a381fddb06a.png","width":1396,"height":268},"locale":"ja","path":"/content/papers/ja/graph-learning/deeper-insights.dba605607ed9.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"全結合ネットワークと比べる","text":"全結合ネットワークと比べる","depth":3},{"id":"ラプラシアン平滑化","text":"ラプラシアン平滑化","depth":3},{"id":"平滑化しすぎると","text":"平滑化しすぎると","depth":3},{"id":"浅すぎても困る","text":"浅すぎても困る","depth":3},{"id":"対策gcn-にもっとラベルを見せる","text":"対策：GCN にもっとラベルを見せる","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/appnp","relation":"referenced","topic":""},{"id":"graph-learning/sgc","relation":"referenced","topic":""}]},{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"老いた駿馬、志はいまだ遠く","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"ja","path":"/content/papers/ja/graph-learning/gnn-plus.b29a25afa6e6.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"二つのタスク","text":"二つのタスク","depth":3},{"id":"公平な比較か","text":"公平な比較か","depth":3},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの古典モデル","text":"三つの古典モデル","depth":3},{"id":"六つの工夫","text":"六つの工夫","depth":3},{"id":"比較方法","text":"比較方法","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"全体結果","text":"全体結果","depth":3},{"id":"速度もよい","text":"速度もよい","depth":3},{"id":"各工夫の役割","text":"各工夫の役割","depth":3},{"id":"著者が挙げる制約","text":"著者が挙げる制約","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]}],"text":"Graph Transformer は小分子から、百万単位のノード分類へ広がりました。NodeFormer、SGFormer、Polynormer の論文では、GCN、GAT、GraphSAGE が大きく後れを取ります。\n\n北京航空航天大学の Yuankai Luo、Lei Shi、香港理工大学の Xiao-Ming Wu は、新モデルの多くも、明示・暗黙に GNN の局所表現と大域注意を組み合わせていると気付きました。\n\n従来の伝播は、過小評価されているのでしょうか？\n\nWu には馴染みの問いです。2018 年には Qimai Li らと GCN をラプラシアン平滑化と説明し、深さによる over-smoothing を指摘しました。\n\n\n> [!TIP]\n> \n> GCN の有効性と深さの制約は、こちらのノートをどうぞ。\n> \n> - **[[18.01] Deeper Insights：ならしすぎたグラフ](/blog/papers/graph-learning/deeper-insights)**\n\n\n## 問題設定\n\nノード分類は、少数ラベルから同一グラフの残りを予測します。グラフを二つに分けます。\n\n- **同類接続**（homophily）：引用論文のように、つながるノードが同クラスになりやすいグラフ。\n- **異類接続**（heterophily）：つながるノードが異クラスになりやすいグラフ。\n\nGNN は同類接続を仮定するため異類接続が苦手とされますが、標準 GCN も悪くないという研究があります。\n\n従来の再評価は、セット数と多様性、調整が不足すると考え、大規模に、調整だけでどこまで届くかを調べます。\n\n## 解決方法\n\n### 三つの古典モデル\n\nGCN は次数で自分と隣を平均し、GraphSAGE は自分と隣の平均に別の重み、GAT は注意で取り分を決めます。モデル自体は変えません。\n\n### 四つのハイパーパラメーター\n\nほかのネットワークで普通に使う工夫を調整します。\n\n- **正規化**：各層の活性化前に BatchNorm か LayerNorm。\n- **Dropout**：活性化後に適用。\n- **残差接続**：前層に重み行列 $W_r$ を掛け、直接加えます。\n\n$$\nh_v^{l}=\\mathrm{Dropout}\\Big(\\sigma\\Big(\\mathrm{Norm}\\Big(h_v^{l-1}W_r^l+\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l\\Big)\\Big)\\Big)\n$$\n\n- **層数**：1〜10 層を試します。\n\n残差は異類接続で特に有効です。六ノードの環で隣が必ず別クラス、特徴量がクラスの one-hot という極端な例を見ます。\n\n```python\nimport numpy as np\n\nn = 6\nM = np.zeros((n, n))\nfor i in range(n):\n    M[i, (i + 1) % n] = M[(i + 1) % n, i] = 1      # 六ノードの環\nlabel = np.array([0, 1, 0, 1, 0, 1])               # 隣は必ず異クラス\nX = np.eye(2)[label]                               # 特徴量は自分のクラス\n\nAt = M + np.eye(n)\nd = At.sum(1)\nA_hat = At / np.sqrt(np.outer(d, d))               # GCN の加重平均\n\ngcn = A_hat @ X                                    # 近傍の加重平均のみ\nres = X + A_hat @ X                                # 元の自分を加える（W_r は単位行列）\nprint(\"ノード 0、GCN 一層後:\", gcn[0].round(2), \" 正解\", (gcn.argmax(1) == label).sum(), \"個\")\nprint(\"ノード 0、残差追加後:\", res[0].round(2), \" 正解\", (res.argmax(1) == label).sum(), \"個\")\n```\n\n```text\nノード 0、GCN 一層後: [0.33 0.67]  正解 0 個\nノード 0、残差追加後: [1.33 0.67]  正解 6 個\n```\n\n異クラスの隣二つの平均が自分を覆いますが、残差で元情報が残ります。模式例で、実際の $W_r$ は学びます。\n\n### 実験設定\n\n18 データセットです。\n\n- **同類接続** 8 個：Cora、CiteSeer、PubMed、Computer、Photo、CS、Physics、WikiCS。\n- **異類接続** 6 個：Squirrel、Chameleon、Amazon-Ratings、Roman-Empire、Minesweeper、Questions。\n- **大規模** 4 個：ogbn-proteins、ogbn-arxiv、ogbn-products、SNS の pokec。13 万〜245 万ノードです。\n\n大規模向けの SGFormer、Polynormer、GOAT、NodeFormer、NAGphormer と、GraphGPS、Exphormer を比べます。異類接続には専用 H2GCN なども加えます。\n\nPolynormer と同じ探索で、学習率 {0.001, 0.005, 0.01}、幅 {64, 256, 512}、正規化（BN、LN）と残差の有無、dropout {0.2, 0.3, 0.5, 0.7}、1〜10 層、最大 2,500 epoch です。Transformer も同探索・環境で再訓練し、5 初期化平均です。\n\n## 議論\n\n18 中 17 で、調整した古典 GNN が首位です。三分類で見ます。\n\n### 同類接続\n\nGCN は Cora 81.60% から 85.10%、Computer 89.65% から 93.99% へ。GAT の WikiCS は 4.16 ポイント増の 81.07% で、七位から Polynormer を超え首位です。Cora、CiteSeer、PubMed、Physics は調整 GCN が首位です。\n\n### 異類接続\n\nGCN は Squirrel 6.34、Chameleon 4.98 ポイント増で首位、Minesweeper、Questions も首位です。専用の H2GCN、GloGNN は後方に並びます。\n\nRoman-Empire は 73.69% から 91.27% へ、17.58 ポイントも増えます。唯一の例外で、Polynormer 92.66% が最高です。\n\n### 大規模グラフ\n\n| モデル | ogbn-proteins（ROC-AUC） | ogbn-arxiv | ogbn-products | pokec |\n| --- | ---: | ---: | ---: | ---: |\n| GraphGPS | 77.15 | 71.23 | メモリ不足 | メモリ不足 |\n| Exphormer | 77.62 | 72.32 | メモリ不足 | メモリ不足 |\n| SGFormer | 79.92 | 72.76 | 81.54 | 82.44 |\n| Polynormer | 79.53 | 73.40 | 83.82 | 86.06 |\n| GCN（調整後） | 77.29 | **73.53** | 82.33 | **86.33** |\n| GraphSAGE（調整後） | 82.21 | 73.00 | **83.89** | 85.97 |\n| GAT（調整後） | **85.01** | 73.30 | 80.99 | 86.19 |\n\nTransformer も同条件の再訓練です。245 万の ogbn-products、163 万の pokec は GraphGPS、Exphormer がメモリ不足です。四つとも古典 GNN が首位で、pokec の GCN は元報告より 10.88、proteins の GAT は 12.99 ポイント増です。\n\n大グラフのノード表現でも伝播は有効で、既存 Transformer も over-smoothing や長距離依存を真に解いてはいない、と解釈します。\n\n### 何が鍵か\n\n四つを一つずつ除き、四点に整理します。\n\n1. **正規化は大グラフで重要**で、小グラフでは小さい効果です。proteins で除くと SAGE、GAT が 4.79、4.69 ポイント減ります。大グラフの特徴分布のばらつきを安定化するためと推測します。\n2. **Dropout はほぼ常に有効**。除くと SAGE は PubMed で 2.70、Roman-Empire で 6.57 ポイント減ります。\n3. **残差は異類接続で最大の効果**。Roman-Empire の GCN は 16.43 ポイント落ち、91.27% から 74.84% に戻り、改善のほぼ全てを失います。\n4. **異類接続では、より深いネットワークが必要なことが多い**。\n\n[![原論文 Figure 1：横軸 2〜10 層の六折れ線図。上は GCN、下は GraphSAGE。左から同類接続（Computer、Photo、CS）、異類接続（Roman-Empire、Minesweeper、Questions）、大規模（ogbn-arxiv）。同類は 6 層付近で横ばいか減少、異類はおおむね増加、arxiv は 4〜5 層で最大になり下がる。](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)\n\n*原論文 Figure 1（Luo ら）、[**PDF 9 ページ**](https://arxiv.org/pdf/2406.08993v2#page=9)より。原キャプションは省略。*\n\n中央の異類接続は三線ともおおむね上昇し、左の同類接続は多くが 6 層前に頭打ちです。複雑な関係には深さが要り、その訓練を残差が支えると考えます。\n\n## 結論\n\n新モデルはなく、もっとも古い三 GNN を調整し直しました。18 中 17 首位、最大二グラフでは一部 Transformer は実行さえできません。\n\n翌年、同じ著者はグラフ全体を対象にするタスクで GCN、GIN、GatedGCN に六技巧を加え、再び比較します。\n\n\n> [!TIP]\n> \n> グラフレベルの再比較は、こちらのノートをどうぞ。\n> \n> - **[[25.02] GNN+：老いた駿馬、志はいまだ遠く](/blog/papers/graph-learning/gnn-plus)**\n\n\n新 Transformer が GCN に大勝したら、付録で残差、正規化、試した深さを先に見てから、表を信じるか考えましょう。\n","imageSizes":{"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png":{"width":1080,"height":516}}}