{"notes":[{"id":"lightweight/starnet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"StarNet","subtitle":"An Asterisk","author":"Z. Yuan","paperDate":"2024-03-29","publishedOn":"2026-02-04","world":"external/starnet","paperTitle":"Rewrite the Stars","paperUrl":"https://arxiv.org/abs/2403.19967","bibliography":{"title":"Rewrite the Stars","authors":["Xu Ma","Xiyang Dai","Yue Bai","Yizhou Wang","Yun Fu"],"url":"https://arxiv.org/abs/2403.19967","pdfUrl":"https://arxiv.org/pdf/2403.19967v1","codeUrl":"https://github.com/ma-xu/Rewrite-the-Stars","preprint":{"id":"2403.19967","submittedOn":"2024-03-29","url":"https://arxiv.org/abs/2403.19967"},"publication":{"venue":"CVPR","year":2024,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2024/html/Ma_Rewrite_the_Stars_CVPR_2024_paper.html"},"citation":"@misc{lightweightstarnet2024,\n  title = {{Rewrite the Stars}},\n  author = {Xu Ma and Xiyang Dai and Yue Bai and Yizhou Wang and Yun Fu},\n  year = {2024},\n  eprint = {2403.19967},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2403.19967}\n}\n","citationPath":"/content/papers/citations/lightweight/starnet.11711ddcb0cd.bib"},"thumbnail":{"src":"/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png","width":1011,"height":588},"locale":"en","path":"/content/papers/en/lightweight/starnet.7ee7b9efc11d.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"how-much-separates-addition-and-multiplication","text":"How Much Separates Addition and Multiplication?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"expand-the-product","text":"Expand the Product","depth":3},{"id":"even-without-activation-functions","text":"Even Without Activation Functions","depth":3},{"id":"starnet","text":"StarNet","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"where-to-put-multiplication","text":"Where to Put Multiplication","depth":3},{"id":"the-cost-of-speed","text":"The Cost of Speed","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/fasternet","relation":"referenced","topic":""}]},{"id":"lightweight/mobilenet-v2","category":"lightweight","legacyPath":"lightweight/1801-mobilenet-v2","legacySlug":"1801-mobilenet-v2","legacyPaths":[],"title":"MobileNet-V2","subtitle":"Refining the Bottleneck","author":"Z. Yuan","paperDate":"2018-01-13","publishedOn":"2024-06-27","paperTitle":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","paperUrl":"https://arxiv.org/abs/1801.04381","bibliography":{"title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","authors":["Mark Sandler","Andrew Howard","Menglong Zhu","Andrey Zhmoginov","Liang-Chieh Chen"],"url":"https://arxiv.org/abs/1801.04381","pdfUrl":"https://arxiv.org/pdf/1801.04381","codeUrl":null,"preprint":{"id":"1801.04381","submittedOn":"2018-01-13","url":"https://arxiv.org/abs/1801.04381"},"publication":{"venue":"CVPR","year":2018,"kind":"conference","url":"https://arxiv.org/abs/1801.04381"},"citation":"@misc{lightweightmobilenetv22018,\n  title = {{MobileNetV2: Inverted Residuals and Linear Bottlenecks}},\n  author = {Mark Sandler and Andrew Howard and Menglong Zhu and Andrey Zhmoginov and Liang-Chieh Chen},\n  year = {2018},\n  eprint = {1801.04381},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/1801.04381}\n}\n","citationPath":"/content/papers/citations/lightweight/mobilenet-v2.a882707cbaa2.bib"},"thumbnail":{"src":"/content/papers/assets/e6bbfb346bb449f83897d76b69c3711d2e9937d40320a7aab0f3843653a688c0.jpg","width":2332,"height":1324},"locale":"en","path":"/content/papers/en/lightweight/mobilenet-v2.db02333fcec2.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"nonlinearity-in-narrow-channels","text":"Nonlinearity in Narrow Channels","depth":3},{"id":"where-does-the-residual-shortcut-connect","text":"Where Does the Residual Shortcut Connect?","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"inverted-residual-and-linear-bottleneck","text":"Inverted Residual and Linear Bottleneck","depth":3},{"id":"why-expand-the-bottleneck","text":"Why Expand the Bottleneck?","depth":3},{"id":"from-block-to-network","text":"From Block to Network","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet-classification-and-phone-latency","text":"ImageNet Classification and Phone Latency","depth":3},{"id":"other-tasks-and-limits","text":"Other Tasks and Limits","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v1","relation":"background","topic":"Efficient convolution on mobile devices"},{"id":"lightweight/mobilenet-v3","relation":"next","topic":"Efficient convolution on mobile devices"}]},{"id":"lightweight/fasternet","category":"lightweight","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"FasterNet","subtitle":"FLOPs Lie","author":"Z. Yuan","paperDate":"2023-03-07","publishedOn":"2025-10-23","world":"external/fasternet","paperTitle":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","paperUrl":"https://arxiv.org/abs/2303.03667","bibliography":{"title":"Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks","authors":["Jierun Chen","Shiu-hong Kao","Hao He","Weipeng Zhuo","Song Wen","Chul-Ho Lee","S.-H. Gary Chan"],"url":"https://arxiv.org/abs/2303.03667","pdfUrl":"https://arxiv.org/pdf/2303.03667v3","codeUrl":null,"preprint":{"id":"2303.03667","submittedOn":"2023-03-07","url":"https://arxiv.org/abs/2303.03667"},"publication":{"venue":"CVPR","year":2023,"kind":"conference","url":"https://openaccess.thecvf.com/content/CVPR2023/html/Chen_Run_Dont_Walk_Chasing_Higher_FLOPS_for_Faster_Neural_Networks_CVPR_2023_paper.html"},"citation":"@misc{lightweightfasternet2023,\n  title = {{Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks}},\n  author = {Jierun Chen and Shiu-hong Kao and Hao He and Weipeng Zhuo and Song Wen and Chul-Ho Lee and S.-H. Gary Chan},\n  year = {2023},\n  eprint = {2303.03667},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2303.03667}\n}\n","citationPath":"/content/papers/citations/lightweight/fasternet.7a313d25fdbe.bib"},"thumbnail":{"src":"/content/papers/assets/f7b24b9388f5bb090f5420a96d640e9f851fd40becb24e6bb9b31ba1ebee4b71.png","width":1012,"height":500},"locale":"en","path":"/content/papers/en/lightweight/fasternet.c960f39f4c31.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"latency-is-flops-divided-by-flops","text":"Latency Is FLOPs Divided by FLOPS","depth":3},{"id":"why-depthwise-convolution-is-slow","text":"Why Depthwise Convolution Is Slow","depth":3},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"partial-convolution-pconv","text":"Partial Convolution (PConv)","depth":3},{"id":"followed-by-a-11-convolution","text":"Followed by a 1×1 Convolution","depth":3},{"id":"fasternet","text":"FasterNet","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"imagenet","text":"ImageNet","depth":3},{"id":"detection-and-segmentation","text":"Detection and Segmentation","depth":3},{"id":"ablation","text":"Ablation","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"lightweight/mobilenet-v2","relation":"referenced","topic":""},{"id":"lightweight/shufflenet","relation":"referenced","topic":""},{"id":"lightweight/ghostnet","relation":"referenced","topic":""}]}],"text":"An elementwise multiplication is often tucked inside the new architectures of recent years.\n\nReplace a Transformer's FFN with a GLU and two branches multiply. Mamba has a gating branch. In vision, FocalNet, HorNet, and VAN multiply too. The multiplication symbol `*` looks like a star, so this paper calls it the “star operation.”\n\nEach paper finds multiplication useful but offers a different explanation. FocalNet describes modulation or gating, HorNet points to higher-order features, and VAN calls it convolutional attention. This paper, from Northeastern University and Microsoft, tries to make the mechanism clear. What do two multiplied linear branches add?\n\n## Defining the Problem\n\n### How Much Separates Addition and Multiplication?\n\nThe authors start with a simple block. Input passes through LayerNorm and depthwise convolution, then splits into two fully connected branches, one with GELU. The branches combine by addition or multiplication, followed by a fully connected layer and a residual connection.\n\n[![Figure 1 from the paper: on the left, a demonstration block runs from Layer Norm to DW-Conv, then splits into two FC branches, with GELU on the left branch. They merge at a red question mark labeled sum or star, then pass through an FC layer and residual addition. The bar chart on the right plots network width from 96 to 448 horizontally, with light-blue sum and light-orange star ImageNet top-1 bars. A dashed red line with triangles marks the gap, decreasing from about 6.5 percentage points at width 96 to about 2.5 at 448.](/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png)](/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png)\n\n*Figure 1 from Ma et al., taken from [**PDF page 1**](https://arxiv.org/pdf/2403.19967v1#page=1). The original caption is omitted.*\n\nStack 12 such blocks to make DemoNet. Change only how the branches combine and compare on ImageNet:\n\n| Network width | 96 | 128 | 160 | 192 | 224 | 256 | 288 |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| Addition | 51.1% | 58.5% | 62.7% | 66.2% | 69.6% | 71.4% | 72.5% |\n| Multiplication | **57.6%** | **64.0%** | **68.2%** | **71.7%** | **73.9%** | **75.3%** | **76.1%** |\n\n*Table: from the paper's Table 2, at fixed depth 12.*\n\nMultiplication wins at every width, with a larger gain in narrower networks: 6.5 percentage points at width 96, falling to 3.6 at 288. The authors also fix width and vary depth; the gap stays between 4 and 6.5 points rather than shrinking with depth.\n\nMultiplication is particularly useful in narrow networks. That observation will matter later.\n\n## Solving the Problem\n\n### Expand the Product\n\nFirst consider one output channel. Input $x$ has $d$ channels, with a 1 appended so the bias can be folded into the weights. Multiplying the two branches gives\n\n$$\n(w_1^\\top x)(w_2^\\top x)=\\Big(\\sum_{i=1}^{d+1} w_1^i x_i\\Big)\\Big(\\sum_{j=1}^{d+1} w_2^j x_j\\Big)=\\sum_{i=1}^{d+1}\\sum_{j=1}^{d+1} w_1^i w_2^j\\, x_i x_j .\n$$\n\nThe expansion is a linear combination of all $x_i x_j$ terms. Since $x_i x_j$ and $x_j x_i$ are identical, merging them leaves $(d+2)(d+1)/2$ terms, roughly $d^2/2$. Computation takes place in $d$ dimensions, yet the result acts like a linear operation in a space of roughly $d^2/2$ dimensions, each a quadratic term of the input.\n\nVerify this at $d=4$:\n\n```python\nimport numpy as np\n\nrng = np.random.default_rng(0)\nd = 4\nx = np.append(rng.normal(size=d), 1.0)        # Append 1 so the bias can be folded into the weights\nw1, w2 = rng.normal(size=d + 1), rng.normal(size=d + 1)\n\nstar = (w1 @ x) * (w2 @ x)                    # Elementwise product of two linear branches\n\n# Expand into a linear combination of all x_i x_j terms\nA = np.outer(w1, w2)\nterms = {(i, j): (A[i, i] if i == j else A[i, j] + A[j, i]) for i in range(d + 1) for j in range(i, d + 1)}\nexpanded = sum(a * x[i] * x[j] for (i, j), a in terms.items())\n\nprint(f\"Direct product: {star:.6f}\")\nprint(f\"Quadratic expansion: {expanded:.6f}\")\nprint(\"Input dimension\", d, \", expanded terms\", len(terms))\nsym = (A + A.T) / 2                           # These 15 coefficients come from a symmetric matrix\nprint(\"Coefficient matrix rank:\", np.linalg.matrix_rank(sym))\n```\n\n```text\nDirect product: -0.059305\nQuadratic expansion: -0.059305\nInput dimension 4 , expanded terms 15\nCoefficient matrix rank: 2\n```\n\nFour input channels expand into 15 quadratic terms. At width 128, there are 8385 terms.\n\nThis resembles the polynomial kernel in machine learning. The polynomial kernel $k(x_1,x_2)=(\\gamma\\, x_1\\cdot x_2+c)^d$ computes in a high-dimensional polynomial feature space without explicitly expanding the features. The star operation similarly computes in low dimensions to obtain a high-dimensional representation.\n\nStack more layers and dimensionality keeps growing through repeated doubling. Multiplying the first layer's quadratic terms again in the second layer gives fourth-order terms. The authors calculate an implicit dimensionality of roughly $90^{1024}$ for a width-128, 10-layer network, effectively infinite for practical purposes.\n\nThe final output line deserves a pause, though. Fifteen coefficients may sound like many, but they all come from two vectors, $w_1$ and $w_2$, producing a coefficient matrix of rank only 2. One output channel cannot choose quadratic coefficients arbitrarily. The dimension is high, but the coefficients are tightly constrained. In the discussion, the authors also note that coefficients in the implicit dimensions cannot be adjusted as freely as ordinary network weights. This may explain why extremely high implicit dimensionality yields only moderate gains.\n\n[![Figure 2 from the paper: four two-dimensional scatterplots show orange and green points arranged as interlocking crescents. The top row contains neural networks: sum operation on the left gives a slanted straight decision boundary, while star operation on the right produces a curved boundary that follows the crescents. The bottom row contains SVMs: the Gaussian Kernel on the left gives a rounded green region, while the Polynomial Kernel on the right resembles the star-operation boundary above.](/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png)](/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png)\n\n*Figure 2 from Ma et al., taken from [**PDF page 5**](https://arxiv.org/pdf/2403.19967v1#page=5). The original caption is omitted.*\n\nThe authors plot decision boundaries on two-moons data. The additive network draws a straight line; the multiplicative network draws a curve resembling a polynomial-kernel SVM more closely than a Gaussian-kernel SVM.\n\n### Even Without Activation Functions\n\nIf multiplication itself supplies nonlinearity, what happens with no activation functions anywhere? Try XOR with a tiny activation-free network. Quadrants one and three form one class; quadrants two and four form the other. No straight line can separate them:\n\n```python\nimport torch\n\ntorch.manual_seed(0)\nx = torch.rand(2000, 2) * 2 - 1                # Points in the plane, with coordinates between -1 and 1\ny = (x[:, 0] * x[:, 1] > 0).float()           # Quadrants 1 and 3 are 1; quadrants 2 and 4 are 0 (XOR)\n\ndef train(combine, width=4):\n    W1, W2 = torch.nn.Linear(2, width), torch.nn.Linear(2, width)\n    head = torch.nn.Linear(width, 1)\n    params = [*W1.parameters(), *W2.parameters(), *head.parameters()]\n    opt = torch.optim.Adam(params, lr=0.05)\n    for _ in range(1000):\n        logit = head(combine(W1(x), W2(x))).squeeze(1)\n        loss = torch.nn.functional.binary_cross_entropy_with_logits(logit, y)\n        opt.zero_grad()\n        loss.backward()\n        opt.step()\n    return ((logit > 0).float() == y).float().mean().item()\n\nprint(f\"Add two branches: accuracy {train(lambda a, b: a + b):.1%}\")\nprint(f\"Multiply two branches: accuracy {train(lambda a, b: a * b):.1%}\")\n```\n\n```text\nAdd two branches: accuracy 53.6%\nMultiply two branches: accuracy 100.0%\n```\n\nThe additive version stays linear from end to end and gets about half right. The multiplicative version includes the $x_1x_2$ term and solves the task directly.\n\nThe paper tries the same on DemoNet at width 192 and depth 12. Removing every activation drops the additive network from 66.2% to 32.4%, while the multiplicative network falls only from 71.7% to 70.5%.\n\n### StarNet\n\nSince multiplication helps most in narrow networks, the authors build a lightweight network around it, called StarNet.\n\n[![Figure 3 from the paper: the overall StarNet pipeline at the top takes a starry-sky image through Conv (BN, ReLU6), then stage1 through stage4. Each stage contains a Conv and a stack of Star Blocks; channels increase from d to 8d and resolution falls from H/4 to H/32, followed by GAP and FC. The lower-left star block has DW-Conv and BN, then two FC branches expanding d to 4d, one with ReLU6. They multiply at a star, pass through FC to return to d, then BN and DW-Conv, with an outer residual connection. The lower-right legend specifies Conv ks=3, stride=2; DW-Conv ks=7, stride=1; the star means elementwise multiplication.](/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png)](/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png)\n\n*Figure 3 from Ma et al., taken from [**PDF page 6**](https://arxiv.org/pdf/2403.19967v1#page=6). The original caption is omitted.*\n\nThe architecture is deliberately plain. Four stages each begin with a stride-2, 3×3 convolution to downsample and double channels. The blocks modify DemoNet in a few ways. BatchNorm replaces LayerNorm and moves after the depthwise convolution so it can be merged at inference. GELU becomes MobileNetV2's ReLU6. Another depthwise convolution is added at the block's end, and both branches expand channels by 4.\n\nThe authors use none of the familiar extras—reparameterization, attention, or SE—to avoid obscuring the star operation's contribution. Four sizes vary only the first-stage channel count and blocks per stage. The smallest S1 has 2.9M parameters and 425M FLOPs; the largest S4 has 7.5M and 1075M.\n\n\n> [!TIP]\n> \n> For ReLU6 and inverted residuals, see MobileNetV2. For another lightweight network emphasizing actual speed, see FasterNet:\n> \n> - **[[18.01] MobileNet-V2: Refining the Bottleneck](/blog/papers/lightweight/mobilenet-v2)**\n> - **[[23.03] FasterNet: FLOPs Lie](/blog/papers/lightweight/fasternet)**\n\n\n## Discussion\n\n### ImageNet\n\nTraining follows the DeiT recipe for 300 epochs. Batch-1 latency is measured on an iPhone 13, a P100 GPU, and an Intel E5-2680 CPU.\n\n| Model | top-1 | Parameters | FLOPs | iPhone 13 | GPU | CPU |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: |\n| MobileOne-S0 | 71.4% | 2.1M | 275M | 0.7 ms | 1.1 ms | 2.2 ms |\n| FasterNet-T0 | 71.9% | 3.9M | 338M | 0.7 ms | 2.5 ms | 5.7 ms |\n| StarNet-S1 | **73.5%** | 2.9M | 425M | 0.7 ms | 2.3 ms | 4.3 ms |\n| MobileOne-S1 | 75.9% | 4.8M | 825M | 0.9 ms | 1.5 ms | 6.0 ms |\n| EfficientNet-B0 | 77.1% | 5.3M | 390M | 1.6 ms | 3.4 ms | 8.8 ms |\n| StarNet-S3 | **77.3%** | 5.8M | 757M | 0.9 ms | 2.7 ms | 6.7 ms |\n| MobileOne-S2 | 77.4% | 7.8M | 1299M | 1.0 ms | 2.0 ms | 8.9 ms |\n| EdgeViT-XS | 77.5% | 6.8M | 1166M | 3.5 ms | 12.1 ms | 18.3 ms |\n| StarNet-S4 | **78.4%** | 7.5M | 1075M | 1.0 ms | 3.7 ms | 9.4 ms |\n\n*Table: selected rows from the paper's Table 6.*\n\nStarNet looks good on the iPhone 13. At the same 0.7 ms, S1 gains 2.1 percentage points over MobileOne-S0. S4 gains 0.9 points over EdgeViT-XS and runs 3.5 times as fast.\n\nGPU and CPU results differ. StarNet-S4 takes 3.7 ms on the P100 against MobileOne-S2's 2.0 ms; on CPU, 9.4 versus 8.9 ms also favors MobileOne slightly. StarNet's strength lies on phones. It is much faster than hybrid architectures such as EdgeViT on all three devices, while its comparison with MobileOne depends on the device.\n\n### Where to Put Multiplication\n\nThe authors replace multiplication with addition in selected StarNet-S4 stages:\n\n| Stage 1 | Stage 2 | Stage 3 | Stage 4 | top-1 |\n| --- | --- | --- | --- | ---: |\n| Add | Add | Add | Add | 75.3% |\n| Multiply | Multiply | Add | Add | 75.2% |\n| Multiply | Multiply | Multiply | Add | 76.8% |\n| Add | Add | Multiply | Multiply | **78.4%** |\n| Multiply | Multiply | Multiply | Multiply | **78.4%** |\n\n*Table: selected rows from the paper's Table 7.*\n\nReplacing all multiplication costs 3.1 percentage points. Yet multiplication in the first two stages makes little difference: using it only in the last two matches using it throughout. The authors suggest that the first two stages are too narrow and ReLU6 zeros many features, also zeroing their quadratic products, so the high-dimensional benefit cannot emerge.\n\nThey also test activation placement. No activations gives 75.6%; one branch gives 78.4%; both branches give 78.0%; activation after multiplication gives 77.0%. Applying it to just one branch works best.\n\nTransforming only one branch and leaving the other as the input $x$, written $(W_2\\,\\mathrm{act}(W_1 x)) * x$, drops accuracy to 74.4% at matched compute. Both branches need linear transformations to generate enough quadratic terms in the expansion.\n\n### The Cost of Speed\n\nMultiplication is theoretically a little costlier than addition. Measurements find equal speed on iPhone and GPU; multiplication is slightly slower on CPU, with StarNet-S4 taking 9.4 versus 8.4 ms.\n\nThere is another issue the paper leaves open. When [a GitHub user asks about quantization](https://github.com/ma-xu/Rewrite-the-Stars/issues/10), the author replies that it has not been tried. Another user notes that multiplying two features can push values outside the INT8 range. That step needs your own validation before phone deployment.\n\n## Conclusion\n\nThis paper turns a widely used operation with competing explanations into one readable expansion. Multiplying two linear branches forms a linear combination of roughly $d^2/2$ quadratic terms; stacking layers keeps increasing the implicit dimension. It also explains why multiplication helps narrow networks most: dimension is exactly what they lack.\n\nThe rank-2 coefficient matrix shows the constraint as well. There are many expanded terms but few freely adjustable coefficients. In DemoNet and StarNet, “near-infinite dimensions” translate into gains of 3 to 6.5 percentage points.\n\ntimm provides pretrained weights from `starnet_s1` to `starnet_s4`. If your lightweight model needs better accuracy, find an addition sign in its blocks and try replacing it with multiplication.\n","imageSizes":{"/content/papers/assets/57e0e49bab986c607a82db36bbc0a45a49e42732719480f1afed603ca4c3e48a.png":{"width":1011,"height":588},"/content/papers/assets/b55872c7135a393362fc7046bb8e6019d674440f9d2e201f4eae032c743d3b02.png":{"width":978,"height":609},"/content/papers/assets/55ab4184389c408d6a5688d8bb59a97d8b20245cb9681ca5344bd295116806f6.png":{"width":1011,"height":693}}}