{"notes":[{"id":"lightweight/starnet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"StarNet","subtitle":"一颗星号","author":"Z. Yuan","paperDate":"2024-03-29","publishedOn":"2026-02-04","world":"external/starnet","paperTitle":"Rewrite the Stars","paperUrl":"https://arxiv.org/abs/2403.19967","bibliography":{"title":"Rewrite the Stars","authors":["Xu Ma","Xiyang Dai","Yue Bai","Yizhou Wang","Yun Fu"],"url":"https://arxiv.org/abs/2403.19967","pdfUrl":"https://arxiv.org/pdf/2403.19967v1","codeUrl":"https://github.com/ma-xu/Rewrite-the-Stars","preprint":{"id":"2403.19967","submittedOn":"2024-03-29","url":"https://arxiv.org/abs/2403.19967"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2024/html/Ma_Rewrite_the_Stars_CVPR_2024_paper.html"},"citation":"@misc{lightweightstarnet2024,\n  title = {{Rewrite the Stars}},\n  author = {Xu Ma and Xiyang Dai and Yue Bai and Yizhou Wang and Yun Fu},\n  year = {2024},\n  eprint = {2403.19967},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2403.19967}\n}\n","citationPath":"/content/papers/citations/lightweight/starnet.11711ddcb0cd.bib"},"thumbnail":{"src":"/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png","width":1011,"height":588},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/starnet.3ca8df96c6ca.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"加和乘差多少","text":"加和乘差多少","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"把乘法展开","text":"把乘法展开","depth":3},{"id":"拿掉激活函数也行","text":"拿掉激活函数也行","depth":3},{"id":"starnet","text":"StarNet","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"乘法放在哪里","text":"乘法放在哪里","depth":3},{"id":"速度的代价","text":"速度的代价","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/fasternet","relation":"referenced","topic":""}]},{"id":"lightweight/mobilenet-v2","category":"lightweight","legacyPath":"lightweight/1801-mobilenet-v2","legacySlug":"1801-mobilenet-v2","legacyPaths":[],"title":"MobileNet-V2","subtitle":"瓶颈内的淬炼","author":"Z. Yuan","paperDate":"2018-01-13","publishedOn":"2024-06-27","paperTitle":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","paperUrl":"https://arxiv.org/abs/1801.04381","bibliography":{"title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","authors":["Mark Sandler","Andrew Howard","Menglong Zhu","Andrey Zhmoginov","Liang-Chieh Chen"],"url":"https://arxiv.org/abs/1801.04381","pdfUrl":"https://arxiv.org/pdf/1801.04381","codeUrl":null,"preprint":{"id":"1801.04381","submittedOn":"2018-01-13","url":"https://arxiv.org/abs/1801.04381"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://arxiv.org/abs/1801.04381"},"citation":"@misc{lightweightmobilenetv22018,\n  title = {{MobileNetV2: Inverted Residuals and Linear Bottlenecks}},\n  author = {Mark Sandler and Andrew Howard and Menglong Zhu and Andrey Zhmoginov and Liang-Chieh Chen},\n  year = {2018},\n  eprint = {1801.04381},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.04381}\n}\n","citationPath":"/content/papers/citations/lightweight/mobilenet-v2.a882707cbaa2.bib"},"thumbnail":{"src":"/content/papers/assets/e6bbfb346bb449f83897d76b69c3711d2e9937d40320a7aab0f3843653a688c0.jpg","width":2332,"height":1324},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/mobilenet-v2.3cc14e69626b.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"线性瓶颈","text":"线性瓶颈","depth":3},{"id":"残差网络","text":"残差网络","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"反转残差","text":"反转残差","depth":3},{"id":"扩张瓶颈论文附录-a","text":"扩张瓶颈（论文附录 A）","depth":3},{"id":"模型架构","text":"模型架构","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"图像分类性能","text":"图像分类性能","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v1","relation":"background","topic":"移动设备上的高效卷积"},{"id":"lightweight/mobilenet-v3","relation":"next","topic":"移动设备上的高效卷积"}]},{"id":"lightweight/fasternet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FasterNet","subtitle":"FLOPs 会骗人","author":"Z. Yuan","paperDate":"2023-03-07","publishedOn":"2025-10-23","world":"external/fasternet","paperTitle":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","paperUrl":"https://arxiv.org/abs/2303.03667","bibliography":{"title":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","authors":["Jierun Chen","Shiu-hong Kao","Hao He","Weipeng Zhuo","Song Wen","Chul-Ho Lee","S.-H. Gary Chan"],"url":"https://arxiv.org/abs/2303.03667","pdfUrl":"https://arxiv.org/pdf/2303.03667v3","codeUrl":null,"preprint":{"id":"2303.03667","submittedOn":"2023-03-07","url":"https://arxiv.org/abs/2303.03667"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Chen_Run_Dont_Walk_Chasing_Higher_FLOPS_for_Faster_Neural_Networks_CVPR_2023_paper.html"},"citation":"@misc{lightweightfasternet2023,\n  title = {{Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks}},\n  author = {Jierun Chen and Shiu-hong Kao and Hao He and Weipeng Zhuo and Song Wen and Chul-Ho Lee and S.-H. Gary Chan},\n  year = {2023},\n  eprint = {2303.03667},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2303.03667}\n}\n","citationPath":"/content/papers/citations/lightweight/fasternet.7a313d25fdbe.bib"},"thumbnail":{"src":"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png","width":1012,"height":500},"locale":"zh-hans","path":"/content/papers/zh-hans/lightweight/fasternet.46f8910dbb1b.md","headings":[{"id":"定义问题","text":"定义问题","depth":2},{"id":"延迟等于-flops-除以-flops","text":"延迟等于 FLOPs 除以 FLOPS","depth":3},{"id":"depthwise-卷积慢在哪里","text":"depthwise 卷积慢在哪里","depth":3},{"id":"解决问题","text":"解决问题","depth":2},{"id":"部分卷积-pconv","text":"部分卷积 PConv","depth":3},{"id":"再接一个-11-卷积","text":"再接一个 1×1 卷积","depth":3},{"id":"fasternet","text":"FasterNet","depth":3},{"id":"讨论","text":"讨论","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"检测与分割","text":"检测与分割","depth":3},{"id":"消融","text":"消融","depth":3},{"id":"结论","text":"结论","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/shufflenet","relation":"referenced","topic":""},{"id":"lightweight/ghostnet","relation":"referenced","topic":""}]}],"text":"这几年的新架构里，常常藏着一个逐元素相乘。\n\nTransformer 的 FFN 换成 GLU，两条分支相乘；Mamba 的区块里有一条门控分支；视觉这边的 FocalNet、HorNet、VAN 也都在乘。乘法的符号 `*` 长得像星星，这篇论文就把它叫做「星号运算」（star operation）。\n\n每篇论文都说乘法有用，理由却各说各话。FocalNet 说它像调制或门控，HorNet 说它带来高阶特征，VAN 说它是一种卷积式注意力。这篇来自美国东北大学（Northeastern University）和微软，作者想把这件事说清楚。两条线性分支乘起来，到底多了什么？\n\n## 定义问题\n\n### 加和乘差多少\n\n作者先做了一个很单纯的区块。输入过 LayerNorm 和 depthwise 卷积，分成两条全连接分支，其中一条接 GELU，两条再用加法或乘法合起来，最后接一个全连接层和残差。\n\n[![原论文 Figure 1：左边是示范用的区块，由上到下是 Layer Norm、DW-Conv，接着分成两个 FC 分支，左边那条带 GELU，两条在红色问号处合并，问号旁标注 sum or star，再接一个 FC 和残差相加。右边是长条图，横轴是网络宽度 96 到 448，浅蓝色长条是 sum 的 ImageNet top-1，浅橘色是 star，红色三角形虚线是两者差距，从宽度 96 的约 6.5 个百分点一路降到 448 的约 2.5。](/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png)](/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png)\n\n*原论文 Figure 1（Ma 等人），取自 [**PDF 第 1 页**](https://arxiv.org/pdf/2403.19967v1#page=1)。原图注未收录。*\n\n把这个区块叠 12 层，叫做 DemoNet，只改合并的方式，在 ImageNet 上比较：\n\n| 网络宽度 | 96 | 128 | 160 | 192 | 224 | 256 | 288 |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 相加 | 51.1% | 58.5% | 62.7% | 66.2% | 69.6% | 71.4% | 72.5% |\n| 相乘 | **57.6%** | **64.0%** | **68.2%** | **71.7%** | **73.9%** | **75.3%** | **76.1%** |\n\n*表：摘自原论文 Table 2，深度固定 12 层。*\n\n每一种宽度都是乘法赢，而且网络越窄赢越多，宽度 96 时差了 6.5 个百分点，到 288 剩 3.6。作者也试了固定宽度、改变深度，差距一直维持在 4 到 6.5 个百分点，没有随深度缩小。\n\n乘法在窄网络上特别有用，这点后面会用到。\n\n## 解决问题\n\n### 把乘法展开\n\n先看一个输出通道。输入 $x$ 有 $d$ 个通道，最后补一个 1，让偏置项也能并入权重。两条分支相乘就是\n\n$$\n(w_1^\\top x)(w_2^\\top x)=\\Big(\\sum_{i=1}^{d+1} w_1^i x_i\\Big)\\Big(\\sum_{j=1}^{d+1} w_2^j x_j\\Big)=\\sum_{i=1}^{d+1}\\sum_{j=1}^{d+1} w_1^i w_2^j\\, x_i x_j .\n$$\n\n展开之后是所有 $x_i x_j$ 的线性组合。$x_i x_j$ 和 $x_j x_i$ 是同一项，合并后共有 $(d+2)(d+1)/2$ 项，大约是 $d^2/2$。换句话说，计算是在 $d$ 维做的，结果却像是在一个约 $d^2/2$ 维的空间里做线性运算，而且每一维都是输入的二次项。\n\n我们用 $d=4$ 验证一次：\n\n```python\nimport numpy as np\n\nrng = np.random.default_rng(0)\nd = 4\nx = np.append(rng.normal(size=d), 1.0)        # 最后补一个 1，偏置项就能并入权重\nw1, w2 = rng.normal(size=d + 1), rng.normal(size=d + 1)\n\nstar = (w1 @ x) * (w2 @ x)                    # 两个线性分支逐元素相乘\n\n# 展开成所有 x_i x_j 的线性组合\nA = np.outer(w1, w2)\nterms = {(i, j): (A[i, i] if i == j else A[i, j] + A[j, i]) for i in range(d + 1) for j in range(i, d + 1)}\nexpanded = sum(a * x[i] * x[j] for (i, j), a in terms.items())\n\nprint(f\"直接相乘：{star:.6f}\")\nprint(f\"展开成二次项：{expanded:.6f}\")\nprint(\"输入维度\", d, \"，展开后的项数\", len(terms))\nsym = (A + A.T) / 2                           # 这 15 个系数其实来自一个对称矩阵\nprint(\"系数矩阵的秩：\", np.linalg.matrix_rank(sym))\n```\n\n```text\n直接相乘：-0.059305\n展开成二次项：-0.059305\n输入维度 4 ，展开后的项数 15\n系数矩阵的秩： 2\n```\n\n4 个输入通道，展开成 15 个二次项。宽度 128 的话是 8385 项。\n\n这和机器学习里的多项式核很像。多项式核 $k(x_1,x_2)=(\\gamma\\, x_1\\cdot x_2+c)^d$ 不用真的把特征展开，就能在高维的多项式特征空间里做计算。星号运算也是在低维计算，换到高维的表示。\n\n叠更多层，维度会继续翻倍增长。第二层再把第一层的二次项乘一次，就变成四次项。作者算过，宽度 128、叠 10 层的网络，隐含的维度大约是 $90^{1024}$，可以当成无限大。\n\n不过最后一行输出值得停一下。15 个系数看起来很多，可是它们全部来自 $w_1$ 和 $w_2$ 这两个向量，组成的系数矩阵秩只有 2。一个输出通道没办法任意挑选要哪些二次项，维度虽然高，系数却被绑得很紧。作者在讨论里也提到，隐含维度上的系数分布没办法像一般网络的权重那样自由调整，这可能是「极高的隐含维度只换来中等提升」的原因。\n\n[![原论文 Figure 2：四张二维散点图，橘色和绿色两群点排成两个交错的月牙形。上排是神经网络，左边 sum operation 的分界是一条斜直线，右边 star operation 的分界是弯曲的，比较贴合月牙。下排是 SVM，左边 Gaussian Kernel 的绿色区域是一团圆弧，右边 Polynomial Kernel 的分界形状和上排的 star operation 很像。](/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png)](/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png)\n\n*原论文 Figure 2（Ma 等人），取自 [**PDF 第 5 页**](https://arxiv.org/pdf/2403.19967v1#page=5)。原图注未收录。*\n\n作者在双月牙数据上画了决策边界。相加的网络画出一条直线，相乘的网络画出曲线，形状和多项式核的 SVM 很接近，和高斯核的 SVM 反而不像。\n\n### 拿掉激活函数也行\n\n如果乘法本身就带来非线性，那网络里完全不放激活函数会怎样？我们用一个没有任何激活函数的小网络试试 XOR，第一、三象限是一类，第二、四象限是另一类，任何一条直线都切不开：\n\n```python\nimport torch\n\ntorch.manual_seed(0)\nx = torch.rand(2000, 2) * 2 - 1                # 平面上的点，坐标在 -1 到 1 之间\ny = (x[:, 0] * x[:, 1] > 0).float()           # 第一、三象限是 1，第二、四象限是 0（XOR）\n\ndef train(combine, width=4):\n    W1, W2 = torch.nn.Linear(2, width), torch.nn.Linear(2, width)\n    head = torch.nn.Linear(width, 1)\n    params = [*W1.parameters(), *W2.parameters(), *head.parameters()]\n    opt = torch.optim.Adam(params, lr=0.05)\n    for _ in range(1000):\n        logit = head(combine(W1(x), W2(x))).squeeze(1)\n        loss = torch.nn.functional.binary_cross_entropy_with_logits(logit, y)\n        opt.zero_grad()\n        loss.backward()\n        opt.step()\n    return ((logit > 0).float() == y).float().mean().item()\n\nprint(f\"两个分支相加：准确率 {train(lambda a, b: a + b):.1%}\")\nprint(f\"两个分支相乘：准确率 {train(lambda a, b: a * b):.1%}\")\n```\n\n```text\n两个分支相加：准确率 53.6%\n两个分支相乘：准确率 100.0%\n```\n\n相加的版本从头到尾都是线性函数，只能猜个一半；相乘的版本里有 $x_1x_2$ 这一项，直接解决。\n\n论文在 DemoNet 上做了同样的事，宽度 192、深度 12。所有激活函数拿掉后，相加的网络从 66.2% 掉到 32.4%，相乘的网络从 71.7% 只掉到 70.5%。\n\n### StarNet\n\n既然乘法在窄网络上效果最好，作者就拿它做轻量网络，叫 StarNet。\n\n[![原论文 Figure 3：上方是 StarNet 的整体流程，一张星空图片输入后经过 Conv（BN、ReLU6），再依序是 stage1 到 stage4，每个 stage 是一个 Conv 接一叠 Star Blocks，通道数从 d 到 8d、分辨率从 H/4 到 H/32，最后接 GAP 和 FC。左下角放大一个 star block：DW-Conv、BN 之后分成两个 FC 分支，把通道从 d 放大到 4d，其中一条带 ReLU6，两条在星号处相乘，再接 FC 缩回 d、BN、DW-Conv，外面一条残差。右下角图例写着 Conv 的 ks=3、stride=2，DW-Conv 的 ks=7、stride=1，星号是逐元素相乘。](/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png)](/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png)\n\n*原论文 Figure 3（Ma 等人），取自 [**PDF 第 6 页**](https://arxiv.org/pdf/2403.19967v1#page=6)。原图注未收录。*\n\n架构刻意做得很朴素。四个阶段，每个阶段开头用步长 2 的 3×3 卷积降采样、通道加倍。区块就是刚才的 DemoNet 区块再改几处。LayerNorm 换成 BatchNorm，放在 depthwise 卷积后面，推理时可以并入卷积；GELU 换成 MobileNetV2 用的 ReLU6；区块尾端再多一个 depthwise 卷积；两条分支都把通道放大到 4 倍。\n\n重参数化、注意力、SE 这些常见的加分技巧一个都没有用，作者说是怕盖过星号运算本身的贡献。四个尺寸只改第一阶段的通道数和每阶段的区块数，最小的 S1 有 2.9M 参数、425M FLOPs，最大的 S4 有 7.5M、1075M。\n\n\n> [!TIP]\n> \n> ReLU6 和倒置残差可以看 MobileNetV2；同样主打「实际速度」的轻量网络，可以看 FasterNet：\n> \n> - **[[18.01] MobileNet-V2：瓶颈内的淬炼](/blog/papers/lightweight/mobilenet-v2)**\n> - **[[23.03] FasterNet：FLOPs 会骗人](/blog/papers/lightweight/fasternet)**\n\n\n## 讨论\n\n### ImageNet\n\n训练照 DeiT 的配方，300 个 epoch。延迟在 iPhone 13、P100 GPU 和 Intel E5-2680 CPU 三种设备上用批量 1 量。\n\n| 模型 | top-1 | 参数量 | FLOPs | iPhone 13 | GPU | CPU |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: |\n| MobileOne-S0 | 71.4% | 2.1M | 275M | 0.7 ms | 1.1 ms | 2.2 ms |\n| FasterNet-T0 | 71.9% | 3.9M | 338M | 0.7 ms | 2.5 ms | 5.7 ms |\n| StarNet-S1 | **73.5%** | 2.9M | 425M | 0.7 ms | 2.3 ms | 4.3 ms |\n| MobileOne-S1 | 75.9% | 4.8M | 825M | 0.9 ms | 1.5 ms | 6.0 ms |\n| EfficientNet-B0 | 77.1% | 5.3M | 390M | 1.6 ms | 3.4 ms | 8.8 ms |\n| StarNet-S3 | **77.3%** | 5.8M | 757M | 0.9 ms | 2.7 ms | 6.7 ms |\n| MobileOne-S2 | 77.4% | 7.8M | 1299M | 1.0 ms | 2.0 ms | 8.9 ms |\n| EdgeViT-XS | 77.5% | 6.8M | 1166M | 3.5 ms | 12.1 ms | 18.3 ms |\n| StarNet-S4 | **78.4%** | 7.5M | 1075M | 1.0 ms | 3.7 ms | 9.4 ms |\n\n*表：摘自原论文 Table 6 的部分行。*\n\n在 iPhone 13 上，StarNet 确实漂亮。同样 0.7 ms，S1 比 MobileOne-S0 高 2.1 个百分点；S4 比 EdgeViT-XS 高 0.9，速度是它的 3.5 倍。\n\n换到 GPU 和 CPU，情况就不太一样。StarNet-S4 在 P100 上要 3.7 ms，MobileOne-S2 只要 2.0 ms；CPU 上 9.4 对 8.9，也是 MobileOne 快一点。StarNet 的强项在手机；和 EdgeViT 这类混合架构比，三种设备上都快不少，和 MobileOne 比就要看设备。\n\n### 乘法放在哪里\n\n作者把 StarNet-S4 里的乘法一段一段换回加法：\n\n| 第 1 阶段 | 第 2 阶段 | 第 3 阶段 | 第 4 阶段 | top-1 |\n| --- | --- | --- | --- | ---: |\n| 加 | 加 | 加 | 加 | 75.3% |\n| 乘 | 乘 | 加 | 加 | 75.2% |\n| 乘 | 乘 | 乘 | 加 | 76.8% |\n| 加 | 加 | 乘 | 乘 | **78.4%** |\n| 乘 | 乘 | 乘 | 乘 | **78.4%** |\n\n*表：摘自原论文 Table 7 的部分行。*\n\n全部换成加法掉了 3.1 个百分点，可是前两个阶段用不用乘法几乎没差，只要后两个阶段用乘法，就和全部用乘法一样。作者的解释是前两个阶段太窄，ReLU6 把不少特征切成 0，乘起来的二次项也跟着变成 0，高维的好处发挥不出来。\n\n激活函数的位置也试了一轮。完全不放激活函数是 75.6%，只在一条分支放是 78.4%，两条都放是 78.0%，乘完再放是 77.0%。只动一条分支最好。\n\n如果只转换一条分支，另一条直接用输入 $x$，写成 $(W_2\\,\\mathrm{act}(W_1 x)) * x$，计算量调到一样，准确率掉到 74.4%。两条分支都要经过线性转换，展开后的二次项才够多。\n\n### 速度的代价\n\n乘法理论上比加法贵一点。实际量起来，iPhone 和 GPU 上两者一样快；CPU 上乘法慢了一些，StarNet-S4 是 9.4 对 8.4 ms。\n\n还有一个论文没处理的问题。GitHub 上[有用户问到量化](https://github.com/ma-xu/Rewrite-the-Stars/issues/10)，作者回复还没试过；另一位用户留言提醒，两个特征相乘，数值范围容易超出 INT8 能表示的范围。部署到手机之前，这一步要自己验证。\n\n## 结论\n\n这篇把一个大家都在用、却各有说法的运算，改写成一行看得懂的展开式。两条线性分支相乘，等于在约 $d^2/2$ 个二次项上做线性组合，层数一叠，隐含维度就一路往上翻。这也解释了为什么乘法在窄网络上赢最多，窄网络最缺的就是维度。\n\n那个秩只有 2 的系数矩阵也说明了上限在哪里。展开的项很多，能自由调的系数却很少，「接近无限维」在 DemoNet 和 StarNet 上换来的是 3 到 6.5 个百分点。\n\ntimm 里就有 `starnet_s1` 到 `starnet_s4` 的预训练权重。手上的轻量模型不够准的话，不妨先找找区块里哪个加号，改成乘号试试看。\n","imageSizes":{"/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png":{"width":1011,"height":588},"/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png":{"width":978,"height":609},"/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png":{"width":1011,"height":693}}}