{"notes":[{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"Old Steeds, New Ambitions","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"en","path":"/content/papers/en/graph-learning/gnn-plus.e317f7610577.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"two-kinds-of-task","text":"Two Kinds of Task","depth":3},{"id":"is-the-comparison-fair","text":"Is the Comparison Fair?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-classic-models","text":"Three Classic Models","depth":3},{"id":"six-techniques","text":"Six Techniques","depth":3},{"id":"how-to-compare","text":"How to Compare","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"overall-results","text":"Overall Results","depth":3},{"id":"faster-too","text":"Faster Too","depth":3},{"id":"what-each-technique-does","text":"What Each Technique Does","depth":3},{"id":"the-authors-own-limitations","text":"The Authors' Own Limitations","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]},{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"Tune First, Compare Later","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"en","path":"/content/papers/en/graph-learning/classic-gnns.d4fa602fd8f6.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-classic-models","text":"Three Classic Models","depth":3},{"id":"four-hyperparameters","text":"Four Hyperparameters","depth":3},{"id":"experimental-setup","text":"Experimental Setup","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"homophilic-graphs","text":"Homophilic Graphs","depth":3},{"id":"heterophilic-graphs","text":"Heterophilic Graphs","depth":3},{"id":"large-graphs","text":"Large Graphs","depth":3},{"id":"which-hyperparameter-matters-most","text":"Which Hyperparameter Matters Most?","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/graphgps","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GraphGPS","subtitle":"GPS for Graphs","author":"Z. Yuan","paperDate":"2022-05-25","publishedOn":"2025-09-09","world":"external/graphgps","paperTitle":"Recipe for a General, Powerful, Scalable Graph Transformer","paperUrl":"https://arxiv.org/abs/2205.12454","bibliography":{"title":"Recipe for a General, Powerful, Scalable Graph Transformer","authors":["Ladislav Rampášek","Mikhail Galkin","Vijay Prakash Dwivedi","Anh Tuan Luu","Guy Wolf","Dominique Beaini"],"url":"https://arxiv.org/abs/2205.12454","pdfUrl":"https://arxiv.org/pdf/2205.12454v4","codeUrl":"https://github.com/rampasek/GraphGPS","preprint":{"id":"2205.12454","submittedOn":"2022-05-25","url":"https://arxiv.org/abs/2205.12454"},"publication":{"venue":"NeurIPS","year":2022,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2022/hash/5d4834a159f1547b267a05a4e2b7cf5e-Abstract-Conference.html"},"citation":"@misc{graphlearninggraphgps2022,\n  title = {{Recipe for a General, Powerful, Scalable Graph Transformer}},\n  author = {Ladislav Rampášek and Mikhail Galkin and Vijay Prakash Dwivedi and Anh Tuan Luu and Guy Wolf and Dominique Beaini},\n  year = {2022},\n  eprint = {2205.12454},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2205.12454}\n}\n","citationPath":"/content/papers/citations/graph-learning/graphgps.053e58fd5309.bib"},"thumbnail":{"src":"/content/papers/assets/e4177a894582643d7811bced6f6c6cf8bb51ff7738da4af5d98115883e1a4bb8.png","width":1116,"height":798},"locale":"en","path":"/content/papers/en/graph-learning/graphgps.88c624c18f31.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"position-and-structure","text":"Position and Structure","depth":3},{"id":"why-message-passing-needs-them","text":"Why Message Passing Needs Them","depth":3},{"id":"message-passing-before-attention-can-lose-information-first","text":"Message Passing before Attention Can Lose Information First","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-ingredients","text":"Three Ingredients","depth":3},{"id":"one-gps-layer","text":"One GPS Layer","depth":3},{"id":"linear-complexity","text":"Linear Complexity","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"ablation-each-paths-contribution","text":"Ablation: Each Path's Contribution","depth":3},{"id":"comparison-with-other-models","text":"Comparison with Other Models","depth":3},{"id":"reassessment-a-year-later","text":"Reassessment a Year Later","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/graph-transformer","relation":"referenced","topic":""},{"id":"graph-learning/san","relation":"referenced","topic":""},{"id":"graph-learning/graphormer","relation":"referenced","topic":""}]},{"id":"graph-learning/where-did-the-gap-go","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Where Did the Gap Go?","subtitle":"Where Did the Gap Go?","author":"Z. Yuan","paperDate":"2023-09-01","publishedOn":"2025-11-12","paperTitle":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","paperUrl":"https://arxiv.org/abs/2309.00367","bibliography":{"title":"Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark","authors":["Jan Tönshoff","Martin Ritzert","Eran Rosenbluth","Martin Grohe"],"url":"https://arxiv.org/abs/2309.00367","pdfUrl":"https://arxiv.org/pdf/2309.00367v2","codeUrl":"https://github.com/toenshoff/LRGB","preprint":{"id":"2309.00367","submittedOn":"2023-09-01","url":"https://arxiv.org/abs/2309.00367"},"publication":{"venue":"Transactions on Machine Learning Research","shortVenue":"TMLR","year":2024,"kind":"journal","url":"https://openreview.net/forum?id=Nm0WX86sKv"},"citation":"@misc{graphlearningwheredidthegapgo2023,\n  title = {{Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark}},\n  author = {Jan Tönshoff and Martin Ritzert and Eran Rosenbluth and Martin Grohe},\n  year = {2023},\n  eprint = {2309.00367},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2309.00367}\n}\n","citationPath":"/content/papers/citations/graph-learning/where-did-the-gap-go.288771073197.bib"},"thumbnail":{"src":"/content/papers/assets/2381a94a5409a041eaab5b06f8dd33ebdd448087d689d89d780486cf3b900353.png","width":1118,"height":416},"locale":"en","path":"/content/papers/en/graph-learning/where-did-the-gap-go.d761c8afe4b4.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"basic-hyperparameter-search","text":"Basic Hyperparameter Search","depth":3},{"id":"channel-wise-normalization","text":"Channel-Wise Normalization","depth":3},{"id":"filtered-mrr","text":"Filtered MRR","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"peptide-datasets","text":"Peptide Datasets","depth":3},{"id":"one-prediction-head","text":"One Prediction Head","depth":3},{"id":"superpixel-datasets","text":"Superpixel Datasets","depth":3},{"id":"pcqm-contact","text":"PCQM-Contact","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/lrgb","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]}],"text":"After GraphGPS, Graph Transformers nearly fill leaderboards for graph classification and regression. Papers repeat a familiar account: message passing GNNs face expressiveness limits, over-smoothing with depth and over-squashing over distance; global attention fills these gaps.\n\nYuankai Luo, Lei Shi and Xiao-Ming Wu at Beihang and Hong Kong Polytechnic University are skeptical. The previous year, they placed GCN, GAT and GraphSAGE in search spaces as broad as those of new models and matched or beat the best models of the time on 17 of 18 node-classification datasets.\n\n\n> [!TIP]\n> \n> For that node-classification reassessment, see our note:\n> \n> - **[[24.06] Classic GNNs: Tune First, Compare Later](/blog/papers/graph-learning/classic-gnns)**\n\n\nNow they ask the same question for graph-level tasks: can classic GNNs compete with Graph Transformers using today's common training techniques?\n\n## Defining the Problem\n\n### Two Kinds of Task\n\nThe authors first distinguish node-level and graph-level tasks. Node classification usually operates on one large social or citation graph, with training, validation and test sets comprising different nodes. Graph-level tasks use whole graphs as examples, typically many small molecules or proteins, and split by graph.\n\nData composition, splitting and training strategies all differ, so node-classification conclusions cannot simply be transferred to graph-level tasks.\n\n### Is the Comparison Fair?\n\nMost evidence favoring Graph Transformers comes from graph-level tasks, especially small-molecule classification and regression. Yet GNN baselines often retain settings from years earlier, missing techniques now routine in newer models.\n\nHow much of the gap comes from architecture, and how much from undertuned baselines?\n\n## Solving the Problem\n\n### Three Classic Models\n\nThree message passing backbones are selected:\n\n- **GCN**: sum self and neighboring features after symmetric degree normalization, then apply weights.\n- **GIN**: multiply self by $1+\\epsilon$, add neighboring sums and apply an MLP.\n- **GatedGCN**: for each edge, a gate computed from endpoint and edge features controls how much neighboring information passes.\n\n### Six Techniques\n\nAdd six common techniques and call the result GNN+:\n\n1. **Edge features**: multiply $e_{uv}$ by a learnable matrix and include it in messages. Molecular bonds and superpixel spatial relationships live on edges.\n2. **Normalization**: BatchNorm after aggregation and before activation in every layer stabilizes distributions.\n3. **Dropout**: after activation, reducing co-adaptation between neurons.\n4. **Residual connections**: add the layer input directly to its output.\n5. **Feedforward network (FFN)**: add a two-layer FFN per layer, like a Transformer's, with residuals and normalization.\n6. **Positional encoding**: concatenate random-walk structural encodings (RWSE)—the probability of returning to self after $m$ steps—with input features, then apply a linear layer.\n\nWith all techniques, one GCN+ layer is:\n\n$$\nh_v^l=\\mathrm{FFN}\\left(\\mathrm{Dropout}\\left(\\sigma\\left(\\mathrm{BN}\\left(\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l+e_{uv}W_e^l\\right)\\right)\\right)+h_v^{l-1}\\right)\n$$\n\n$$\n\\mathrm{FFN}(h)=\\mathrm{BN}\\left(\\sigma\\left(hW_{\\mathrm{FFN}_1}^l\\right)W_{\\mathrm{FFN}_2}^l+h\\right)\n$$\n\nReading outward: messages include edge features, followed by BatchNorm, ReLU and dropout; add the previous layer's input as a residual, then apply the FFN.\n\n[![Figure 2 of the paper: GNN+ architecture. Node features are concatenated with positional encodings, then enter L repeated blocks alongside edge features. Each block is a GNN layer followed by an FFN, ending in readout. The expanded GNN layer runs message passing, normalization, activation and dropout, then a residual; the FFN contains an MLP followed by Add & Norm.](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)](/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png)\n\n*Figure 2 from Luo et al., taken from [**PDF page 3**](https://arxiv.org/pdf/2502.09263v3#page=3). The original caption is omitted.*\n\nNone of these ingredients is new. Residuals, normalization and dropout come from CNNs; FFNs and positional encoding from Transformers; RWSE is common in Graph Transformers such as GraphGPS. The contribution is their systematic combination and a fair comparison.\n\n### How to Compare\n\nFourteen graph-level datasets fall into three groups:\n\n- **Benchmarking GNNs**: ZINC molecular regression; MNIST and CIFAR10 classification after superpixel conversion; PATTERN and CLUSTER node classification on synthetic graphs.\n- **LRGB**: Peptides-func and Peptides-struct peptide molecules, PascalVOC-SP and COCO-SP superpixel segmentation, plus MalNet-Tiny malware call graphs.\n- **OGB**: ogbg-molhiv, ogbg-molpcba, ogbg-ppa and ogbg-code2.\n\nSearch spaces follow GraphGPS, with benchmark-specific parameter budgets, such as around 500,000 on ZINC. Baseline Graph Transformers are retrained under the same search space and environment. Other baseline numbers come from papers or leaderboards and are mostly already tuned. Every experiment uses 5 seeds.\n\n## Discussion\n\n### Overall Results\n\nEnhanced classic GNNs place in the top three on every one of the 14 datasets, winning 8. Selected numbers:\n\n| Dataset (metric) | Original GCN | GCN+ | GatedGCN+ | GraphGPS | Best other model in table |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| ZINC (MAE ↓) | 0.367 | 0.076 | 0.077 | 0.070 | 0.057 (TIGT) |\n| CIFAR10 (accuracy ↑) | 55.71% | 69.82% | **77.22%** | 72.30% | 76.85% (GRED) |\n| Peptides-func (AP ↑) | 0.6860 | **0.7261** | 0.7006 | 0.6535 | 0.7150 (DRew) |\n| Peptides-struct (MAE ↓) | 0.2460 | **0.2421** | 0.2431 | 0.2500 | 0.2449 (Graph ViT) |\n| PascalVOC-SP (F1 ↑) | 0.2078 | 0.3357 | 0.4263 | 0.4440 | 0.4561 (GSSC) |\n\nSeveral points stand out:\n\n1. **Original and enhanced models differ greatly.** On ZINC, GCN MAE falls from 0.367 to 0.076, GIN from 0.526 to 0.065, with the same backbone and six added techniques.\n2. **The small-molecule gap nearly disappears.** GIN+'s 0.065 on ZINC beats GraphGPS. The best Graph Transformer still leads slightly, but the gap shrinks from severalfold to a few hundredths.\n3. **Classic GNNs lead on long-range datasets.** GCN+'s Peptides-func 0.7261 beats every listed Graph Transformer and Graph Mamba. The “original GCN” LRGB numbers are already Tönshoff et al.'s 2023 tuned results; original LRGB GCN scored only 0.5930.\n4. **They do not win everywhere.** On PascalVOC-SP, GatedGCN+ still trails the 2023 GraphGPS result and GSSC.\n\nOn OGB, GatedGCN+ is top three on all four datasets, winning three. It improves ogbg-ppa by around 9%, topping the OGB leaderboard; on ogbg-molhiv and ogbg-molpcba, it matches Graphormer and EGT that were pretrained elsewhere.\n\n### Faster Too\n\nWithout global attention, classic GNNs require much less computation. Per epoch on ZINC, GraphGPS takes 21 seconds and GCN+ 7. MalNet-Tiny graphs average 1,410 nodes; GraphGPS takes 46 seconds, GCN+ only 6.\n\n### What Each Technique Does\n\nRemoving the six techniques one by one yields six observations:\n\n1. **Edge features matter particularly for molecular and superpixel graphs.** Edges encode bonds and spatial relationships. On code call graphs, edge features are less relevant to the task and removing them matters little.\n2. **Normalization matters more on large datasets.** With 150,000 ogbg-ppa graphs, removing it drops all three models' accuracy by around 15%.\n3. **Dropout usually helps, but only a little is needed.** Of optimal dropout rates, 97% are at most 0.2 and 64% at most 0.1. This contrasts with the authors' node-classification findings, where higher dropout is usually needed.\n4. **Residuals are usually indispensable**, except for shallow networks on small graphs. Peptide models use only 3 to 5 layers and change little without residuals; elsewhere, 10-plus-layer networks often deteriorate sharply.\n5. **FFNs are critical for GCN+ and GIN+.** Removing the FFN from GIN+ on MNIST drops accuracy from 98.29% to 11.35%, an 88% relative decline. GatedGCN+'s gates make it much less dependent on FFNs.\n6. **Positional encoding helps small datasets, less so large ones.** ZINC, PATTERN, CLUSTER, Peptides-func and ogbg-molhiv have only ten to forty thousand graphs and clearly worsen without RWSE. Beyond a hundred thousand graphs, the effect is less obvious. The authors suggest enough data reduces reliance on encodings for structural understanding.\n\nCLUSTER gives the most dramatic ablation. Chance on this 6-class node task is around 16.7%; removing residuals drops all three models to 16.5%–16.8%, and removing normalization does the same to GCN+. In these networks deeper than 10 layers, omitting residuals or normalization leaves the model effectively learning nothing.\n\n\n> [!TIP]\n> \n> For GraphGPS, the reassessment supplying the table's LRGB baselines, and the GCN and GIN backbones, see our notes:\n> \n> - **[[22.05] GraphGPS: GPS for Graphs](/blog/papers/graph-learning/graphgps)**\n> - **[[23.09] Where Did the Gap Go?: Where Did the Gap Go?](/blog/papers/graph-learning/where-did-the-gap-go)**\n> - **[[16.09] GCN: Is One Parameter Enough?](/blog/papers/graph-learning/gcn)**\n> - **[[18.10] GIN: Summing Is What Counts](/blog/papers/graph-learning/gin)**\n\n\n### The Authors' Own Limitations\n\nThe authors acknowledge that their conclusions are entirely empirical. There is no theoretical explanation for why classic GNNs catch up, how the six techniques interact, or whether they truly mitigate over-smoothing and over-squashing.\n\nCiting Bechler-Speicher et al.'s 2025 discussion, they also question whether current datasets reflect the complexity and diversity of real graph problems. More challenging, application-driven benchmarks could change the relative standings again.\n\n## Conclusion\n\nAdding six common techniques to GCN, GIN and GatedGCN lets them compete directly with Graph Transformers across 14 graph-level datasets: top three everywhere, 8 first places, and several times faster training.\n\nRead this alongside its predecessor and Tönshoff et al.'s reassessment. All three ask that a new architecture's advantage be established against equally well-tuned baselines.\n\nAttention remains in the picture. GNN+'s FFNs and RWSE are familiar Graph Transformer components; the two toolboxes have already mixed.\n\nBefore trying a new graph-level model, run GatedGCN+ as your baseline. The authors release [code](https://github.com/LUOyk1999/GNNPlus) and hyperparameters for every dataset, so getting started takes little time.\n","imageSizes":{"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png":{"width":604,"height":510}}}