{"notes":[{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"Tune First, Compare Later","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"en","path":"/content/papers/en/graph-learning/classic-gnns.d4fa602fd8f6.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-classic-models","text":"Three Classic Models","depth":3},{"id":"four-hyperparameters","text":"Four Hyperparameters","depth":3},{"id":"experimental-setup","text":"Experimental Setup","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"homophilic-graphs","text":"Homophilic Graphs","depth":3},{"id":"heterophilic-graphs","text":"Heterophilic Graphs","depth":3},{"id":"large-graphs","text":"Large Graphs","depth":3},{"id":"which-hyperparameter-matters-most","text":"Which Hyperparameter Matters Most?","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/deeper-insights","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Deeper Insights","subtitle":"Smoothed Too Flat","author":"Z. Yuan","paperDate":"2018-01-22","publishedOn":"2023-11-29","paperTitle":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","paperUrl":"https://arxiv.org/abs/1801.07606","bibliography":{"title":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","authors":["Qimai Li","Zhichao Han","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/1801.07606","pdfUrl":"https://arxiv.org/pdf/1801.07606v1","codeUrl":null,"preprint":{"id":"1801.07606","submittedOn":"2018-01-22","url":"https://arxiv.org/abs/1801.07606"},"publication":{"venue":"AAAI","year":2018,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/11604"},"citation":"@misc{graphlearningdeeperinsights2018,\n  title = {{Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning}},\n  author = {Qimai Li and Zhichao Han and Xiao-Ming Wu},\n  year = {2018},\n  eprint = {1801.07606},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.07606}\n}\n","citationPath":"/content/papers/citations/graph-learning/deeper-insights.52354c86fac9.bib"},"thumbnail":{"src":"/content/papers/assets/2a13880b48fae6456e19c622cee1d7c7907f8c31d6890722d4980a381fddb06a.png","width":1396,"height":268},"locale":"en","path":"/content/papers/en/graph-learning/deeper-insights.ce733c5df590.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"compare-with-a-fully-connected-network-first","text":"Compare with a Fully Connected Network First","depth":3},{"id":"laplacian-smoothing","text":"Laplacian Smoothing","depth":3},{"id":"too-much-smoothing","text":"Too Much Smoothing","depth":3},{"id":"too-shallow-is-a-problem-too","text":"Too Shallow Is a Problem Too","depth":3},{"id":"a-remedy-give-gcn-more-labels-to-see","text":"A Remedy: Give GCN More Labels to See","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/appnp","relation":"referenced","topic":""},{"id":"graph-learning/sgc","relation":"referenced","topic":""}]},{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"Old Steeds, New Ambitions","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"en","path":"/content/papers/en/graph-learning/gnn-plus.e317f7610577.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"two-kinds-of-task","text":"Two Kinds of Task","depth":3},{"id":"is-the-comparison-fair","text":"Is the Comparison Fair?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-classic-models","text":"Three Classic Models","depth":3},{"id":"six-techniques","text":"Six Techniques","depth":3},{"id":"how-to-compare","text":"How to Compare","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"overall-results","text":"Overall Results","depth":3},{"id":"faster-too","text":"Faster Too","depth":3},{"id":"what-each-technique-does","text":"What Each Technique Does","depth":3},{"id":"the-authors-own-limitations","text":"The Authors' Own Limitations","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]}],"text":"Graph Transformers first establish themselves on small molecular graphs, then move into node classification, especially graphs with millions of nodes. Papers introducing NodeFormer, SGFormer and Polynormer often leave GCN, GAT and GraphSAGE far behind.\n\nYuankai Luo and Lei Shi at Beihang University and Xiao-Ming Wu at Hong Kong Polytechnic University notice something: many new models still rely, explicitly or implicitly, on a GNN to learn local representations alongside global attention.\n\nHas conventional message passing been underestimated for node classification?\n\nWu knows the question well. In 2018, he, Qimai Li and colleagues explained GCN as Laplacian smoothing and identified over-smoothing at depth.\n\n\n> [!TIP]\n> \n> For why GCN works and why it struggles with depth, see our note:\n> \n> - **[[18.01] Deeper Insights: Smoothed Too Flat](/blog/papers/graph-learning/deeper-insights)**\n\n\n## Defining the Problem\n\nNode classification uses a small set of known labels to predict other nodes' classes on a graph. Two types are distinguished:\n\n- **Homophilic graphs**: connected nodes mostly share a class, as with mutually citing papers.\n- **Heterophilic graphs**: connected nodes often have different classes.\n\nGNNs are generally thought to assume homophily and thus struggle with heterophily, yet recent work finds standard GCN can do quite well there too.\n\nEarlier reassessments, the authors argue, use too few and too narrow a range of datasets and incomplete tuning. This paper makes a sufficiently broad check of how far classic GNNs can go through hyperparameter tuning alone.\n\n## Solving the Problem\n\n### Three Classic Models\n\nThe subjects are three basic GNNs: GCN averages self and neighbors with weights determined by degrees; GraphSAGE weights self and the neighbor mean separately; GAT uses attention to determine each neighbor's share. The models themselves remain unchanged.\n\n### Four Hyperparameters\n\nThe settings are familiar from other neural networks:\n\n- **Normalization**: BatchNorm or LayerNorm before each layer's activation.\n- **Dropout**: after activation.\n- **Residual connections**: multiply the previous representation by $W_r$ and add it directly to this layer:\n\n$$\nh_v^{l}=\\mathrm{Dropout}\\Big(\\sigma\\Big(\\mathrm{Norm}\\Big(h_v^{l-1}W_r^l+\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l\\Big)\\Big)\\Big)\n$$\n\n- **Depth**: try 1 through 10 layers.\n\nResiduals matter especially on heterophilic graphs. Take an extreme six-node cycle whose adjacent nodes always differ in class, with one-hot class features:\n\n```python\nimport numpy as np\n\nn = 6\nM = np.zeros((n, n))\nfor i in range(n):\n    M[i, (i + 1) % n] = M[(i + 1) % n, i] = 1      # Six nodes in a cycle\nlabel = np.array([0, 1, 0, 1, 0, 1])               # Adjacent nodes always have different classes\nX = np.eye(2)[label]                               # Features are the node's own class\n\nAt = M + np.eye(n)\nd = At.sum(1)\nA_hat = At / np.sqrt(np.outer(d, d))               # GCN weighted average\n\ngcn = A_hat @ X                                    # Neighbor-weighted average only\nres = X + A_hat @ X                                # Add the original self representation (identity W_r)\nprint(\"Node 0 after one GCN layer:\", gcn[0].round(2), \" correct\", (gcn.argmax(1) == label).sum(), \"nodes\")\nprint(\"Node 0 with residual:\", res[0].round(2), \" correct\", (res.argmax(1) == label).sum(), \"nodes\")\n```\n\n```text\nNode 0 after one GCN layer: [0.33 0.67]  correct 0 nodes\nNode 0 with residual: [1.33 0.67]  correct 6 nodes\n```\n\nEach node has two different-class neighbors, so averaging lets their class overwhelm self. The residual preserves the original self information. This is only an illustration; the real $W_r$ is learned.\n\n### Experimental Setup\n\nThere are 18 datasets:\n\n- **8 homophilic graphs**: Cora, CiteSeer, PubMed, Computer, Photo, CS, Physics, WikiCS.\n- **6 heterophilic graphs**: Squirrel, Chameleon, Amazon-Ratings, Roman-Empire, Minesweeper, Questions.\n- **4 large graphs**: ogbn-proteins, ogbn-arxiv, ogbn-products and the pokec social network, ranging from 130,000 to 2.45 million nodes.\n\nComparisons include large-graph Transformers SGFormer, Polynormer, GOAT, NodeFormer and NAGphormer, plus GraphGPS and Exphormer. Heterophily-specialized GNNs such as H2GCN are added for heterophilic datasets.\n\nThe search follows Polynormer: learning rate {0.001, 0.005, 0.01}, hidden dimension {64, 256, 512}, optional normalization (BN or LN) and residuals, dropout {0.2, 0.3, 0.5, 0.7}, depth 1 to 10, and at most 2,500 epochs. The authors retrain the Graph Transformers under the same search ranges and training environment. Each result averages 5 initializations.\n\n## Discussion\n\nTuned classic GNNs rank first on 17 of 18 datasets. Consider the three groups separately.\n\n### Homophilic Graphs\n\nTuning raises GCN from 81.60% to 85.10% on Cora and 89.65% to 93.99% on Computer. GAT gains 4.16 percentage points on WikiCS to reach 81.07%, jumping from seventh to first ahead of Polynormer. Tuned GCN wins Cora, CiteSeer, PubMed and Physics.\n\n### Heterophilic Graphs\n\nGains are larger here. GCN rises by 6.34 and 4.98 percentage points on Squirrel and Chameleon, winning both, and also wins Minesweeper and Questions. Specialized H2GCN and GloGNN fall behind.\n\nRoman-Empire is most dramatic: GCN rises from 73.69% to 91.27%, a 17.58-point gain. It is also the sole exception among 18 datasets, with Polynormer still best at 92.66%.\n\n### Large Graphs\n\n| Model | ogbn-proteins (ROC-AUC) | ogbn-arxiv | ogbn-products | pokec |\n| --- | ---: | ---: | ---: | ---: |\n| GraphGPS | 77.15 | 71.23 | Out of memory | Out of memory |\n| Exphormer | 77.62 | 72.32 | Out of memory | Out of memory |\n| SGFormer | 79.92 | 72.76 | 81.54 | 82.44 |\n| Polynormer | 79.53 | 73.40 | 83.82 | 86.06 |\n| GCN (tuned) | 77.29 | **73.53** | 82.33 | **86.33** |\n| GraphSAGE (tuned) | 82.21 | 73.00 | **83.89** | 85.97 |\n| GAT (tuned) | **85.01** | 73.30 | 80.99 | 86.19 |\n\nAll Graph Transformers in the table are retrained under the same conditions. GraphGPS and Exphormer run out of memory on 2.45-million-node ogbn-products and 1.63-million-node pokec. Classic GNNs win all four large graphs. GCN on pokec exceeds its original reported result by 10.88 percentage points; GAT on ogbn-proteins gains 12.99 points.\n\nThe authors interpret this as evidence that message passing remains very effective at learning large-graph node representations. Existing Graph Transformers have yet to truly solve GNNs' old problems of over-smoothing and long-range dependency.\n\n### Which Hyperparameter Matters Most?\n\nRemoving the four settings one at a time gives four observations:\n\n1. **Normalization matters on large graphs**, less on small ones. Without it on ogbn-proteins, GraphSAGE and GAT lose 4.79 and 4.69 percentage points. The authors suggest that varied feature distributions on large graphs make stable normalization useful.\n2. **Dropout almost always helps.** Removing it costs GraphSAGE 2.70 points on PubMed and 6.57 on Roman-Empire.\n3. **Residuals matter most under heterophily.** On Roman-Empire, removing them drops GCN by 16.43 points, from 91.27% to 74.84%. Almost the entire improvement comes from residuals.\n4. **Heterophilic graphs usually need deeper networks.**\n\n[![Figure 1 of the paper: six curves plot depth from 2 to 10 layers. The upper row is GCN, the lower GraphSAGE. Columns show homophilic graphs (Computer, Photo, CS), heterophilic graphs (Roman-Empire, Minesweeper, Questions) and the large ogbn-arxiv graph. Homophilic accuracy levels near 6 layers or falls with depth; heterophilic curves mostly rise; ogbn-arxiv peaks at 4 to 5 then falls.](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)\n\n*Figure 1 from Luo et al., taken from [**PDF page 9**](https://arxiv.org/pdf/2406.08993v2#page=9). The original caption is omitted.*\n\nThe top row is GCN, the bottom GraphSAGE. Middle-column heterophilic curves generally rise with depth; most homophilic curves on the left peak before 6 layers. The authors suggest more complex heterophilic structure requires depth to capture node relationships, while residual connections make those deep networks trainable.\n\n## Conclusion\n\nThis paper proposes no new model. It simply tunes three of the oldest GNNs properly, winning 17 of 18 datasets while some Graph Transformers cannot even run on the largest two graphs.\n\nThe following year, the same authors bring this approach to graph-level tasks, add six common techniques to GCN, GIN and GatedGCN, and compare again with Graph Transformers.\n\n\n> [!TIP]\n> \n> For that graph-level reassessment, see our note:\n> \n> - **[[25.02] GNN+: Old Steeds, New Ambitions](/blog/papers/graph-learning/gnn-plus)**\n\n\nWhen a new Graph Transformer trounces GCN on node classification, check the appendix first: did GCN get residuals and normalization, and how many layers were tried? Then decide how much confidence to place in the table.\n","imageSizes":{"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png":{"width":1080,"height":516}}}