{"notes":[{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"老いた駿馬、志はいまだ遠く","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"ja","path":"/content/papers/ja/graph-learning/gnn-plus.b29a25afa6e6.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"二つのタスク","text":"二つのタスク","depth":3},{"id":"公平な比較か","text":"公平な比較か","depth":3},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの古典モデル","text":"三つの古典モデル","depth":3},{"id":"六つの工夫","text":"六つの工夫","depth":3},{"id":"比較方法","text":"比較方法","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"全体結果","text":"全体結果","depth":3},{"id":"速度もよい","text":"速度もよい","depth":3},{"id":"各工夫の役割","text":"各工夫の役割","depth":3},{"id":"著者が挙げる制約","text":"著者が挙げる制約","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]},{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"調整してから比べよう","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"ja","path":"/content/papers/ja/graph-learning/classic-gnns.2907699f1731.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの古典モデル","text":"三つの古典モデル","depth":3},{"id":"四つのハイパーパラメーター","text":"四つのハイパーパラメーター","depth":3},{"id":"実験設定","text":"実験設定","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"同類接続","text":"同類接続","depth":3},{"id":"異類接続","text":"異類接続","depth":3},{"id":"大規模グラフ","text":"大規模グラフ","depth":3},{"id":"何が鍵か","text":"何が鍵か","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"グラフ上の GPS","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"ja","path":"/content/papers/ja/graph-learning/graphgps.0d5dc280bee1.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"位置と構造","text":"位置と構造","depth":3},{"id":"なぜ伝播にも必要か","text":"なぜ伝播にも必要か","depth":3},{"id":"伝播のあとで注意しても遅い","text":"伝播のあとで注意しても遅い","depth":3},{"id":"解決方法","text":"解決方法","depth":2},{"id":"三つの材料","text":"三つの材料","depth":3},{"id":"gps-の一層","text":"GPS の一層","depth":3},{"id":"線形の計算量","text":"線形の計算量","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"アブレーション二経路の寄与","text":"アブレーション：二経路の寄与","depth":3},{"id":"他モデルとの比較","text":"他モデルとの比較","depth":3},{"id":"一年後の再評価","text":"一年後の再評価","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/where-did-the-gap-go","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Where Did the Gap Go?","subtitle":"差はどこへ消えた？","author":"Z. Yuan","paperDate":"2023-09-01","publishedOn":"2025-11-12","paperTitle":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","paperUrl":"https://arxiv.org/abs/2309.00367","bibliography":{"title":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","authors":["Jan Tönshoff","Martin Ritzert","Eran Rosenbluth","Martin Grohe"],"url":"https://arxiv.org/abs/2309.00367","pdfUrl":"https://arxiv.org/pdf/2309.00367v2","codeUrl":"https://github.com/toenshoff/LRGB","preprint":{"id":"2309.00367","submittedOn":"2023-09-01","url":"https://arxiv.org/abs/2309.00367"},"publication":{"venue":"Transactions on Machine Learning Research","shortVenue":"TMLR","year":2024,"kind":"journal","url":"https://openreview.net/forum?id=Nm0WX86sKv"},"citation":"@misc{graphlearningwheredidthegapgo2023,\n  title = {{Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark}},\n  author = {Jan Tönshoff and Martin Ritzert and Eran Rosenbluth and Martin Grohe},\n  year = {2023},\n  eprint = {2309.00367},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2309.00367}\n}\n","citationPath":"/content/papers/citations/graph-learning/where-did-the-gap-go.288771073197.bib"},"thumbnail":{"src":"/content/papers/assets/2381a94a5409a041eaab5b06f8dd33ebdd448087d689d89d780486cf3b900353.png","width":1118,"height":416},"locale":"ja","path":"/content/papers/ja/graph-learning/where-did-the-gap-go.b479b0d84134.md","headings":[{"id":"問題設定","text":"問題設定","depth":2},{"id":"解決方法","text":"解決方法","depth":2},{"id":"基本的な探索","text":"基本的な探索","depth":3},{"id":"チャネル別正規化","text":"チャネル別正規化","depth":3},{"id":"filtered-mrr","text":"filtered MRR","depth":3},{"id":"議論","text":"議論","depth":2},{"id":"ペプチド","text":"ペプチド","depth":3},{"id":"予測ヘッド一つ","text":"予測ヘッド一つ","depth":3},{"id":"スーパーピクセル","text":"スーパーピクセル","depth":3},{"id":"pcqm-contact","text":"PCQM-Contact","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/lrgb","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]}],"text":"GraphGPS 以降、グラフ分類・回帰のランキングはほぼ Graph Transformer で埋まります。伝播の表現力上限、over-smoothing、over-squashing を大域注意が補う、という説明も共通です。\n\n北京航空航天大学、香港理工大学の Yuankai Luo、Lei Shi、Xiao-Ming Wu は疑います。前年、ノード分類で同じ広さの探索を GCN、GAT、GraphSAGE に行い、18 中 17 で当時最良と同等以上にしたからです。\n\n\n> [!TIP]\n> \n> その再比較は、こちらのノートをどうぞ。\n> \n> - **[[24.06] Classic GNNs：調整してから比べよう](/blog/papers/graph-learning/classic-gnns)**\n\n\n今度はグラフレベルで、現代の訓練の工夫を添えれば、古典 GNN は競えるかを問います。\n\n## 問題設定\n\n### 二つのタスク\n\nノード分類は通常、大きな SNS や引用網の、異なるノードを訓練・検証・テストにします。グラフレベルは多数の小分子やタンパク質を一件ずつ扱い、グラフを分割します。\n\nデータ構成、分割、訓練戦略が異なるので、ノード分類の結論を直接移すことはできません。\n\n### 公平な比較か\n\nTransformer 優位の証拠の多くは、特に小分子分類・回帰のグラフレベルです。しかし GNN は数年前の設定のまま、新モデルで標準の工夫を使っていない例が多くあります。\n\n差のうち、構造そのものと、調整不足がそれぞれどれだけかを調べます。\n\n## 解決方法\n\n### 三つの古典モデル\n\n三伝播モデルを骨組みにします。\n\n- **GCN**：自分と隣を次数で対称正規化して総和し、重みを掛けます。\n- **GIN**：自分の $1+\\epsilon$ 倍と隣の総和を MLP に通します。\n- **GatedGCN**：両端ノードとエッジ特徴量からゲートを作り、通すメッセージ量を決めます。\n\n### 六つの工夫\n\n共通の六つを加え、GNN+ と呼びます。\n\n1. **エッジ特徴量**：$e_{uv}$ に学習行列を掛けて伝播に加えます。分子の結合、スーパーピクセルの空間関係を持ち込みます。\n2. **正規化**：集約後、活性化前に BatchNorm を置き、各層の分布を安定させます。\n3. **Dropout**：活性化後に置き、ニューロン間の共適応を減らします。\n4. **残差接続**：層の入力を出力に直接加えます。\n5. **フィードフォワードネットワーク（FFN）**：Transformer と同じ二層 FFN を各層につなぎ、残差と正規化も使います。\n6. **位置符号化**：$m$ 歩の帰還確率 RWSE を入力に連結し、線形層に通します。\n\n全部を入れた GCN+ の一層は、こうなります。\n\n$$\nh_v^l=\\mathrm{FFN}\\left(\\mathrm{Dropout}\\left(\\sigma\\left(\\mathrm{BN}\\left(\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l+e_{uv}W_e^l\\right)\\right)\\right)+h_v^{l-1}\\right)\n$$\n\n$$\n\\mathrm{FFN}(h)=\\mathrm{BN}\\left(\\sigma\\left(hW_{\\mathrm{FFN}_1}^l\\right)W_{\\mathrm{FFN}_2}^l+h\\right)\n$$\n\n内側から、隣のメッセージにエッジ特徴量を加え、BatchNorm、ReLU、dropout、入力の残差、FFN です。\n\n[![原論文 Figure 2：特徴量に位置符号化を連結し、エッジとともに L 個の GNN 層・FFN ブロックを通して readout する。右の GNN 内部は下から伝播、正規化、活性化と dropout、残差。FFN 内部は MLP と Add & Norm。](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)\n\n*原論文 Figure 2（Luo ら）、[**PDF 3 ページ**](https://arxiv.org/pdf/2502.09263v3#page=3)より。原キャプションは省略。*\n\n新しい部品はありません。残差、正規化、dropout は CNN、FFN と位置は Transformer、RWSE は GraphGPS などから来ています。貢献は系統的な組み合わせと公平な再比較です。\n\n### 比較方法\n\n14 セットを三組にします。\n\n- **Benchmarking GNNs**：ZINC の分子回帰、MNIST、CIFAR10 のスーパーピクセル分類、PATTERN、CLUSTER の合成グラフのノード分類。\n- **LRGB**：ペプチドの func、struct、画像の PascalVOC-SP、COCO-SP と、マルウェア呼び出しグラフの MalNet-Tiny。\n- **OGB**：ogbg-molhiv、ogbg-molpcba、ogbg-ppa、ogbg-code2。\n\nGraphGPS と同じ探索、各規定のパラメーター予算（ZINC 約 50 万パラメーター）です。比較 Transformer も同探索・環境で再訓練し、ほかは論文・ランキングの多くが調整済みの値を使います。5 シードを実行します。\n\n## 議論\n\n### 全体結果\n\n14 全てで強化 GNN が三位以内、うち 8 首位です。いくつか見ます。\n\n| データセット（指標） | 元の GCN | GCN+ | GatedGCN+ | GraphGPS | ほかのモデルの最良 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| ZINC（MAE ↓） | 0.367 | 0.076 | 0.077 | 0.070 | 0.057（TIGT） |\n| CIFAR10（精度 ↑） | 55.71% | 69.82% | **77.22%** | 72.30% | 76.85%（GRED） |\n| Peptides-func（AP ↑） | 0.6860 | **0.7261** | 0.7006 | 0.6535 | 0.7150（DRew） |\n| Peptides-struct（MAE ↓） | 0.2460 | **0.2421** | 0.2431 | 0.2500 | 0.2449（Graph ViT） |\n| PascalVOC-SP（F1 ↑） | 0.2078 | 0.3357 | 0.4263 | 0.4440 | 0.4561（GSSC） |\n\n注目点があります。\n\n1. **元と強化後で大差**。同じ骨組みの六技巧だけで、ZINC は GCN 0.367 から 0.076、GIN 0.526 から 0.065 へ下がります。\n2. **小分子の差はほぼ消える**。GIN+ 0.065 は GPS を超えます。最良 Transformer は少し上でも、数倍差が 0.0 数程度に縮みます。\n3. **長距離でも古典 GNN が先行**。func の GCN+ 0.7261 は全掲載 Transformer、Graph Mamba より上です。「元の GCN」も既に 2023 年 Tönshoff らの調整値で、LRGB 元値は 0.5930 です。\n4. **勝たない項目もある**。PascalVOC-SP の GatedGCN+ は、2023 年版 GPS と GSSC に届きません。\n\nOGB は GatedGCN+ が全四つで三位以内、三つで首位です。ppa は約 9% 上がり OGB 首位、molhiv、molpcba は他データで事前学習した Graphormer、EGT と同等です。\n\n### 速度もよい\n\n大域的注意がなく、計算は小さくなります。一 epoch は ZINC で GPS 21 秒、GCN+ 7 秒。平均 1,410 ノードの MalNet-Tiny は GPS 46 秒、GCN+ 6 秒です。\n\n### 各工夫の役割\n\n一つずつ除いて、六観察にまとめます。\n\n1. **エッジ特徴量は分子・画像で重要**。結合と空間関係があるためです。コード呼び出しグラフは関連が薄く、除いても小さな変化です。\n2. **正規化は大データで重要**。15 万グラフの ppa は、除くと三モデルとも約 15% 下がります。\n3. **Dropout は少量で有効**。最適率の 97% が 0.2 以下、64% が 0.1 以下です。高率が必要だったノード分類と逆だと指摘します。\n4. **残差はほぼ不可欠**。小グラフの浅いモデルは例外です。ペプチドの 3〜5 層では影響が小さく、他の 10 層超では大幅悪化しがちです。\n5. **FFN は GCN+、GIN+ に重要**。MNIST の GIN+ は 98.29% から 11.35%、相対 88% 減です。ゲートを持つ GatedGCN+ は依存が小さくなります。\n6. **位置は小データで有効**。1〜4 万グラフの ZINC、PATTERN、CLUSTER、func、molhiv は RWSE なしで落ちます。十万超では小さな効果で、データが多ければ、構造の理解で符号化に頼る必要が小さくなると推測します。\n\n顕著なのは CLUSTER の六分類です。ランダム約 16.7% に対し、残差を外すと三モデルとも 16.5%〜16.8%、GCN+ は正規化なしでも同じです。10 層超では、残差や正規化を欠くとほぼ学べません。\n\n\n> [!TIP]\n> \n> 比較 GPS、LRGB の再評価、GCN、GIN は、こちらのノートをどうぞ。\n> \n> - **[[22.05] GraphGPS：グラフ上の GPS](/blog/papers/graph-learning/graphgps)**\n> - **[[23.09] Where Did the Gap Go?：差はどこへ消えた？](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[16.09] GCN：一つのパラメーターで足りる？](/blog/papers/graph-learning/gcn)**\n> - **[[18.10] GIN：足し合わせてこそ分かる](/blog/papers/graph-learning/gin)**\n\n\n### 著者が挙げる制約\n\n結論はすべて実験からで、追いつく理論的説明はありません。六技巧の相互作用や、実際に over-smoothing、over-squashing を緩和するかも理論分析が必要です。\n\nBechler-Speicher ら（2025）の議論も引き、既存データが現実の複雑さ・多様性を十分表さない可能性を指摘します。難しく実用に近い基準なら、相対位置はまた変わり得ます。\n\n## 結論\n\nGCN、GIN、GatedGCN に六技巧を加え、14 のグラフレベルのデータセットで正面比較しました。全て三位以内、8 首位で、訓練も数倍速くなりました。\n\n前作、Tönshoff らと合わせて読むとよい論文です。三つとも、新構造の優位はベースラインも同等に調整してから測ろう、と述べます。\n\n注意も引き続き舞台に残っています。FFN と RWSE は Graph Transformer 系の定番で、道具箱はすでに混ざっています。\n\n新モデルを試すなら、先に GatedGCN+ を動かして基準にしましょう。[コード](https://github.com/LUOyk1999/GNNPlus)と全データの設定が公開され、時間も多くかかりません。\n","imageSizes":{"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png":{"width":604,"height":510}}}