{"notes":[{"id":"graph-learning/classic-gnns","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Classic GNNs","subtitle":"調好再比","author":"Z. Yuan","paperDate":"2024-06-13","publishedOn":"2025-11-25","paperTitle":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","paperUrl":"https://arxiv.org/abs/2406.08993","bibliography":{"title":"Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2406.08993","pdfUrl":"https://arxiv.org/pdf/2406.08993v2","codeUrl":"https://github.com/LUOyk1999/tunedGNN","preprint":{"id":"2406.08993","submittedOn":"2024-06-13","url":"https://arxiv.org/abs/2406.08993"},"publication":{"venue":"NeurIPS Datasets and Benchmarks","shortVenue":"NeurIPS D&B","year":2024,"kind":"conference","url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/b10ed15ff1aa864f1be3a75f1ffc021b-Abstract-Datasets_and_Benchmarks_Track.html"},"citation":"@misc{graphlearningclassicgnns2024,\n  title = {{Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2024},\n  eprint = {2406.08993},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2406.08993}\n}\n","citationPath":"/content/papers/citations/graph-learning/classic-gnns.166c6c5a9ba0.bib"},"thumbnail":{"src":"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png","width":1080,"height":516},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/classic-gnns.593df559ff7c.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"三個經典模型","text":"三個經典模型","depth":3},{"id":"四個超參數","text":"四個超參數","depth":3},{"id":"實驗設定","text":"實驗設定","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"同質圖","text":"同質圖","depth":3},{"id":"異質圖","text":"異質圖","depth":3},{"id":"大圖","text":"大圖","depth":3},{"id":"哪個超參數最關鍵","text":"哪個超參數最關鍵","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/deeper-insights","relation":"referenced","topic":""},{"id":"graph-learning/gnn-plus","relation":"referenced","topic":""}]},{"id":"graph-learning/deeper-insights","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Deeper Insights","subtitle":"抹得太平了","author":"Z. Yuan","paperDate":"2018-01-22","publishedOn":"2023-11-29","paperTitle":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","paperUrl":"https://arxiv.org/abs/1801.07606","bibliography":{"title":"Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning","authors":["Qimai Li","Zhichao Han","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/1801.07606","pdfUrl":"https://arxiv.org/pdf/1801.07606v1","codeUrl":null,"preprint":{"id":"1801.07606","submittedOn":"2018-01-22","url":"https://arxiv.org/abs/1801.07606"},"publication":{"venue":"AAAI","year":2018,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/11604"},"citation":"@misc{graphlearningdeeperinsights2018,\n  title = {{Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning}},\n  author = {Qimai Li and Zhichao Han and Xiao-Ming Wu},\n  year = {2018},\n  eprint = {1801.07606},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.07606}\n}\n","citationPath":"/content/papers/citations/graph-learning/deeper-insights.52354c86fac9.bib"},"thumbnail":{"src":"/content/papers/assets/2a13880b48fae6456e19c622cee1d7c7907f8c31d6890722d4980a381fddb06a.png","width":1396,"height":268},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/deeper-insights.8ca58f8b9ca8.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"先和全連接網路比","text":"先和全連接網路比","depth":3},{"id":"拉普拉斯平滑","text":"拉普拉斯平滑","depth":3},{"id":"平滑太多次","text":"平滑太多次","depth":3},{"id":"太淺也不行","text":"太淺也不行","depth":3},{"id":"補救讓-gcn-多看到一些標籤","text":"補救：讓 GCN 多看到一些標籤","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/appnp","relation":"referenced","topic":""},{"id":"graph-learning/sgc","relation":"referenced","topic":""}]},{"id":"graph-learning/gnn-plus","category":"graph-learning","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"GNN+","subtitle":"老驥伏櫪","author":"Z. Yuan","paperDate":"2025-02-13","publishedOn":"2025-12-06","paperTitle":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","paperUrl":"https://arxiv.org/abs/2502.09263","bibliography":{"title":"Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence","authors":["Yuankai Luo","Lei Shi","Xiao-Ming Wu"],"url":"https://arxiv.org/abs/2502.09263","pdfUrl":"https://arxiv.org/pdf/2502.09263v3","codeUrl":"https://github.com/LUOyk1999/GNNPlus","preprint":{"id":"2502.09263","submittedOn":"2025-02-13","url":"https://arxiv.org/abs/2502.09263"},"publication":{"venue":"ICML","year":2025,"kind":"conference","url":"https://proceedings.mlr.press/v267/luo25h.html"},"citation":"@misc{graphlearninggnnplus2025,\n  title = {{Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence}},\n  author = {Yuankai Luo and Lei Shi and Xiao-Ming Wu},\n  year = {2025},\n  eprint = {2502.09263},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2502.09263}\n}\n","citationPath":"/content/papers/citations/graph-learning/gnn-plus.7d14d260e648.bib"},"thumbnail":{"src":"/content/papers/assets/0be1b8a606332519d042698f1c6afd3ef8857eb95682b95df1cdeb9e3bfe7648.png","width":604,"height":510},"locale":"zh-hant","path":"/content/papers/zh-hant/graph-learning/gnn-plus.3e66f7f0d156.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"兩種任務","text":"兩種任務","depth":3},{"id":"比較公不公平","text":"比較公不公平","depth":3},{"id":"解決問題","text":"解決問題","depth":2},{"id":"三個經典模型","text":"三個經典模型","depth":3},{"id":"六個技巧","text":"六個技巧","depth":3},{"id":"怎麼比","text":"怎麼比","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"整體結果","text":"整體結果","depth":3},{"id":"也比較快","text":"也比較快","depth":3},{"id":"每個技巧各管什麼","text":"每個技巧各管什麼","depth":3},{"id":"作者自己說的限制","text":"作者自己說的限制","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"graph-learning/classic-gnns","relation":"referenced","topic":""},{"id":"graph-learning/graphgps","relation":"referenced","topic":""},{"id":"graph-learning/where-did-the-gap-go","relation":"referenced","topic":""}]}],"text":"Graph Transformer 先在分子這類小圖上站穩，接著有人把它推向節點分類，尤其是動輒上百萬個節點的大圖。NodeFormer、SGFormer、Polynormer 這些模型的論文裡，GCN、GAT、GraphSAGE 常常被甩在後面。\n\n北京航空航天大學的 Yuankai Luo、Lei Shi 與香港理工大學的 Xiao-Ming Wu 注意到一件事。這些新模型裡，不少明著或暗著，還是靠一個 GNN 去學局部的表示，再搭配全域注意力。\n\n那麼，傳統的訊息傳遞在節點分類上，是不是被低估了？\n\nWu 對這個問題並不陌生。2018 年，他和 Qimai Li 等人把 GCN 解釋成拉普拉斯平滑，也指出疊太深會 over-smoothing。\n\n\n> [!TIP]\n> \n> GCN 為什麼有效、又為什麼疊不深，可以參考我們的筆記：\n> \n> - **[[18.01] Deeper Insights：抹得太平了](/blog/papers/graph-learning/deeper-insights)**\n\n\n## 定義問題\n\n節點分類是在一張圖上，用少數已知標籤的節點，預測其他節點的類別。圖可以分成兩種：\n\n- **同質圖**（homophily）：相連的節點多半同類，例如互相引用的論文。\n- **異質圖**（heterophily）：相連的節點常常不同類。\n\n一般認為 GNN 隱含了同質的假設，所以在異質圖上表現不好；但近年已有研究發現，標準的 GCN 在異質圖上其實也不差。\n\n過去也有人重新評估 GNN，作者認為那些研究的資料集太少、種類不夠多，超參數也沒有調完整。這篇要做一次規模夠大的檢查，看經典的 GNN 只調超參數，能走到哪裡。\n\n## 解決問題\n\n### 三個經典模型\n\n主角是三個最基本的 GNN。GCN 照度數加權平均自己和鄰居；GraphSAGE 把自己和鄰居的平均分開乘權重；GAT 用注意力決定每個鄰居的份量。模型本身一點都不改。\n\n### 四個超參數\n\n要調的，是幾個在其他神經網路上早就用得很習慣的東西：\n\n- **正規化**：在每一層的激活函數之前加 BatchNorm 或 LayerNorm。\n- **Dropout**：加在激活函數之後。\n- **殘差連接**：把上一層的表示乘上一個權重矩陣 $W_r$，直接加進這一層：\n\n$$\nh_v^{l}=\\mathrm{Dropout}\\Big(\\sigma\\Big(\\mathrm{Norm}\\Big(h_v^{l-1}W_r^l+\\sum_{u\\in\\mathcal N(v)\\cup\\{v\\}}\\frac{1}{\\sqrt{\\hat d_u\\hat d_v}}h_u^{l-1}W^l\\Big)\\Big)\\Big)\n$$\n\n- **層數**：從 1 層試到 10 層。\n\n殘差在異質圖上特別有用。用一個極端的例子看，六個節點繞成一圈，相鄰的節點一定不同類，每個節點的特徵就是自己類別的 one-hot：\n\n```python\nimport numpy as np\n\nn = 6\nM = np.zeros((n, n))\nfor i in range(n):\n    M[i, (i + 1) % n] = M[(i + 1) % n, i] = 1      # 六個節點繞成一圈\nlabel = np.array([0, 1, 0, 1, 0, 1])               # 相鄰的節點一定不同類\nX = np.eye(2)[label]                               # 特徵就是自己的類別\n\nAt = M + np.eye(n)\nd = At.sum(1)\nA_hat = At / np.sqrt(np.outer(d, d))               # GCN 的加權平均\n\ngcn = A_hat @ X                                    # 只有鄰居加權平均\nres = X + A_hat @ X                                # 再加上自己原本的表示（W_r 取單位矩陣）\nprint(\"節點 0，GCN 一層後:\", gcn[0].round(2), \" 猜對\", (gcn.argmax(1) == label).sum(), \"個\")\nprint(\"節點 0，加上殘差後:\", res[0].round(2), \" 猜對\", (res.argmax(1) == label).sum(), \"個\")\n```\n\n```text\n節點 0，GCN 一層後: [0.33 0.67]  猜對 0 個\n節點 0，加上殘差後: [1.33 0.67]  猜對 6 個\n```\n\n每個節點有兩個異類的鄰居，平均之後，鄰居的類別蓋過了自己；加上殘差，自己原本的資訊留了下來。這只是示意，實際的 $W_r$ 是學出來的。\n\n### 實驗設定\n\n資料集一共 18 個：\n\n- **同質圖** 8 個：Cora、CiteSeer、PubMed、Computer、Photo、CS、Physics、WikiCS。\n- **異質圖** 6 個：Squirrel、Chameleon、Amazon-Ratings、Roman-Empire、Minesweeper、Questions。\n- **大圖** 4 個：OGB 的 ogbn-proteins、ogbn-arxiv、ogbn-products，以及社群網路 pokec，節點數從 13 萬到 245 萬。\n\n比較的對象包括 SGFormer、Polynormer、GOAT、NodeFormer、NAGphormer 這些為大圖設計的 Graph Transformer，以及 GraphGPS、Exphormer；異質圖上另外加了 H2GCN 等專為異質圖設計的 GNN。\n\n超參數的搜尋範圍沿用 Polynormer，學習率取 {0.001, 0.005, 0.01}，隱藏維度取 {64, 256, 512}，再決定要不要用正規化（BN 或 LN）和殘差，dropout 取 {0.2, 0.3, 0.5, 0.7}，層數從 1 到 10，最多訓練 2,500 個 epoch。作者也用同樣的搜尋範圍和訓練環境，重新訓練了表中的 Graph Transformer；每個結果是 5 次不同初始化的平均。\n\n## 討論\n\n18 個資料集裡，調過的經典 GNN 在 17 個排第一。下面分三類看。\n\n### 同質圖\n\n調參之後，GCN 在 Cora 從 81.60% 升到 85.10%，在 Computer 從 89.65% 升到 93.99%。GAT 在 WikiCS 上升了 4.16 個百分點到 81.07%，從第七名跳到第一，超過原本領先的 Polynormer。Cora、CiteSeer、PubMed、Physics 上，第一名都是調過的 GCN。\n\n### 異質圖\n\n異質圖上的進步更大。GCN 在 Squirrel 和 Chameleon 上分別升了 6.34 和 4.98 個百分點，都排第一；Minesweeper 和 Questions 也是 GCN 第一。H2GCN、GloGNN 這些專為異質圖設計的模型，反而都排在後面。\n\n最誇張的是 Roman-Empire，GCN 從 73.69% 升到 91.27%，多了 17.58 個百分點。這也是 18 個資料集裡唯一的例外，Polynormer 的 92.66% 仍然最高。\n\n### 大圖\n\n| 模型 | ogbn-proteins（ROC-AUC） | ogbn-arxiv | ogbn-products | pokec |\n| --- | ---: | ---: | ---: | ---: |\n| GraphGPS | 77.15 | 71.23 | 記憶體不足 | 記憶體不足 |\n| Exphormer | 77.62 | 72.32 | 記憶體不足 | 記憶體不足 |\n| SGFormer | 79.92 | 72.76 | 81.54 | 82.44 |\n| Polynormer | 79.53 | 73.40 | 83.82 | 86.06 |\n| GCN（調參後） | 77.29 | **73.53** | 82.33 | **86.33** |\n| GraphSAGE（調參後） | 82.21 | 73.00 | **83.89** | 85.97 |\n| GAT（調參後） | **85.01** | 73.30 | 80.99 | 86.19 |\n\n表中的 Graph Transformer 都是作者用同樣條件重新訓練的結果。245 萬個節點的 ogbn-products 和 163 萬個節點的 pokec 上，GraphGPS 和 Exphormer 都記憶體不足。四個大圖上，第一名都是經典 GNN。pokec 上的 GCN 比原本回報的數字多了 10.88 個百分點，ogbn-proteins 上的 GAT 多了 12.99 個百分點。\n\n作者的解讀是，訊息傳遞在大圖上學節點表示依然非常有效，現有的 Graph Transformer 也還沒有真正解決 over-smoothing 和長距離依賴這些 GNN 的老問題。\n\n### 哪個超參數最關鍵\n\n作者把四個超參數一個一個拿掉，看準確率掉多少，整理出四點：\n\n1. **正規化在大圖上很重要**，小圖上影響不大。ogbn-proteins 上拿掉正規化，GraphSAGE 和 GAT 分別掉 4.79 和 4.69 個百分點。作者推測，大圖的節點特徵分布差異大，正規化讓訓練時的分布比較穩定。\n2. **Dropout 幾乎總是有幫助**。拿掉它，GraphSAGE 在 PubMed 掉 2.70、在 Roman-Empire 掉 6.57 個百分點。\n3. **殘差在異質圖上影響最大**。Roman-Empire 上拿掉殘差，GCN 掉了 16.43 個百分點，從 91.27% 掉回 74.84%，GCN 在這個資料集的進步幾乎都來自它。\n4. **異質圖通常需要更深的網路**。\n\n[![原論文 Figure 1：六張折線圖，橫軸是層數 2 到 10。上排是 GCN，下排是 GraphSAGE；左到右分別是同質圖（Computer、Photo、CS）、異質圖（Roman-Empire、Minesweeper、Questions）與大圖（ogbn-arxiv）。同質圖的準確率在 6 層附近持平或隨層數下降；異質圖的三條線大致隨層數上升；ogbn-arxiv 在 4 到 5 層達到最高後下降。](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)](/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png)\n\n*原論文 Figure 1（Luo 等人），取自 [**PDF 第 9 頁**](https://arxiv.org/pdf/2406.08993v2#page=9)。圖說未收錄。*\n\n上排是 GCN，下排是 GraphSAGE。中間那欄的異質圖，三條線大致隨著層數往上走；左邊的同質圖，多數在 6 層之前就到頂了。作者認為，異質圖的結構比較複雜，需要更深的網路才能抓到節點之間的關係，而深網路要訓練得起來，就得靠殘差連接。\n\n## 結論\n\n這篇論文沒有提出新模型，只把三個最老的 GNN 好好調了一次。結果在 18 個資料集裡，它們有 17 個拿第一；在最大的兩張圖上，有些 Graph Transformer 甚至跑不起來。\n\n隔年，同一組作者把同樣的做法搬到圖層級的任務，替 GCN、GIN、GatedGCN 加上六個常見的技巧，再和 Graph Transformer 比了一次。\n\n\n> [!TIP]\n> \n> 圖層級上的重新比較，可以參考我們的筆記：\n> \n> - **[[25.02] GNN+：老驥伏櫪](/blog/papers/graph-learning/gnn-plus)**\n\n\n新的 Graph Transformer 在節點分類上大勝 GCN 時，先翻附錄看看 GCN 有沒有加殘差和正規化、層數試到幾層，再決定要不要相信那張表。\n","imageSizes":{"/content/papers/assets/c0d0343c2ddf281c63e4c48623074c885c2a966b5b3100ad107199638d9822c8.png":{"width":1080,"height":516}}}