MobileOne: An Improved One millisecond Mobile Backbone
- Publication
- CVPR 2023Conference
- First submitted to arXiv
- arXiv:2206.04040
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{reparameterizationmobileone2022,
title = {{MobileOne: An Improved One millisecond Mobile Backbone}},
author = {Pavan Kumar Anasosalu Vasu and James Gabriel and Jeff Zhu and Oncel Tuzel and Anurag Ranjan},
year = {2022},
eprint = {2206.04040},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2206.04040}
}
The concept of reparameterization gained attention around 2021 after the introduction of RepVGG.
Researchers have been exploring ways to find suitable reparameterization methods in model architecture to enhance performance.
Problem Definition
Inspired by the success of RepVGG, the authors aimed to explore the effects of reparameterization by designing another structural unit for reparameterization and conducted neural network deployment and benchmarking on an iPhone 12 using CoreML.
Like RepVGG, MobileOne adds mergeable linear/affine branches during training and folds them together for inference. The network still contains ReLU and is not a linear model. MobileOne also relaxes regularization, including weight decay, during training to avoid over-regularizing small models.
Problem Solving
Bottleneck Analysis - Activation Functions
Table: Paper Table 2. Only the activation changes: ReLU takes 1.53 ms, GELU 1.63 ms, and Dynamic Shift-Max 57.04 ms.
To analyze the impact of activation functions on latency, the authors built a 30-layer convolutional neural network and conducted benchmarking on an iPhone 12 using different activation functions.
The table above shows that despite having identical architectures except for the activation functions, the latencies varied significantly.
These latency differences are mainly due to recently introduced activation functions. Although these functions showed significant accuracy improvements in ultra-low FLOPs models like MicroNet, their usage can incur high latency costs.
Therefore, to reduce latency costs, the authors use ReLU in most MobileOne blocks. The S4 configuration still uses SE-ReLU in its final two stages.
Bottleneck Analysis - Architecture
Table: Paper Table 3. The baseline takes 1.53 ms; adding SE or skip connections separately gives 2.10 or 2.62 ms. These are not cumulative steps.
Next, they considered two common architectures: skip connections and Squeeze-Excite modules.
Two critical factors affecting runtime performance are memory access costs and parallelism. In multi-branch architectures, memory access costs increase significantly as activations for each branch need to be stored to compute the next tensor in the computation graph. If the number of branches is small, this memory bottleneck can be avoided. Additionally, forced synchronization blocks (e.g., global pool operations used in Squeeze-Excite blocks) can also impact overall runtime due to synchronization costs.
To expose memory access and synchronization costs, the authors used the 30-layer network without either module as the baseline, then added skip connections or Squeeze-Excite separately.
Thus, MobileOne adopts an inference architecture with its mergeable branches removed, resulting in smaller memory access costs. Squeeze-Excite modules remain in selected blocks of the largest S4 variant to improve accuracy.
Model Architecture
RepVGG adds mergeable branches to VGG-style 3×3 convolutions; it does not turn arbitrary ResNet residual blocks into single convolutions. MobileOne instead targets MobileNetV1-style depthwise/pointwise blocks.
As shown in the above figure, this time the reparameterization target is the form of depthwise separable convolutions, first performing a 3x3 depthwise convolution, followed by a 1x1 pointwise convolution. Then, reparameterizable skip connections with batch normalization and replicated structure branches are introduced.
At inference, the mergeable additive branches become a single convolution. The SE modules retained in S4 are not among those mergeable branches. For a convolution layer with a kernel size of K, input channel dimension , and output channel dimension , the weight matrix is represented as , and bias as , where is the group count (equal to input channels for depthwise convolution). BN contains accumulated mean , variance , scale , and offset .
First fold BN using fixed statistics, pad smaller kernels to the same size, and represent identity branches as convolutions. Stride, groups and spatial alignment must match. Then, the convolution layer's weight and bias during inference are calculated as follows: weight equals the sum of weights from all branches, i.e., ; bias equals the sum of biases from all branches, i.e., , where is the number of branches.
Model Configuration
Table: Paper Table 7. In each tuple, scales width and counts training branches; S0 often uses , other variants mainly use , and S4 adds SE-ReLU in later stages.
Recent studies scale model width, depth, and resolution. This table lists five S0–S4 configurations that also differ in training branch count and SE placement. The paper does not increase input resolution in this comparison.
MobileOne scales depth similarly to MobileNet-V2, using shallower early stages at higher feature-map resolutions as these layers are much slower than later stages.
Since MobileOne has no multi-branch architecture during inference, it avoids the data movement costs associated with such architectures. This allows for model parameter expansion without significant latency cost increases, which is a major advantage compared to competitive multi-branch architectures like MobileNet-V2 and EfficientNets.
Discussion
ImageNet Performance
Table: Paper Table 9. Classification models use 224 × 224 inputs without distillation. MobileOne-S0/S1 reach 0.79/0.89 ms on the phone; S4 takes 1.86 ms.
The authors evaluated MobileOne models on the ImageNet dataset, which contains 1.28 million training images and 50,000 validation images from 1,000 classes.
All MobileOne variants were trained from scratch using PyTorch on machines equipped with 8 NVIDIA GPUs; the comparison models in the table do not necessarily share this recipe. MobileOne training used SGD with momentum for 300 epochs and an effective batch size of 256. It also used cross-entropy loss with label smoothing of 0.1.
The initial learning rate was set to 0.1 and decayed using a cosine annealing schedule, with the initial weight decay coefficient set to , gradually decaying to .
Larger MobileOne variants (S2, S3, and S4) were trained with AutoAugment, while smaller variants (S0 and S1) used standard data augmentation techniques such as random resized cropping and horizontal flipping.
All versions of MobileOne used EMA (Exponential Moving Average) weights during training with a decay constant of 0.9995.
The main comparison uses 224 × 224 inputs and models trained without distillation. All MobileOne variants have fewer than 20 million parameters, although some additional RepVGG baselines in the table are larger. FLOPs counts were reported using the fvcore library.
In the paper's iPhone 12/CoreML setup, every listed Transformer variant exceeds 4 ms on the phone. Mobile numbers are minimum full round-trip latency after preloading and warm-up; CPU and GPU results use different hardware and statistics. The comparison also depends on the conversion tools and runtime used at the time.
The paper's prose separately cites a MobileFormer variant at 79.3%/70.76 ms and compares it with MobileOne-S4 at 79.4%/1.86 ms, a roughly 38-fold phone-latency ratio; 70.76 ms is not a Mobileformer-52/96 row in Table 9. In Table 9, MobileOne-S3 is 1 percentage point more accurate than EfficientNet-B0, while iPhone 12 latency falls from 1.72 to 1.53 ms, about 11% lower rather than eleven times faster.
Table 9 also reports CPU and GPU latency. Whether MobileOne leads depends on the particular accuracy-matched pair and hardware; not every variant is the fastest on every platform.
Conclusion
The study analyzes efficiency bottlenecks in components used by modern CNNs and shows how reparameterizable structures can ease optimization during training. MobileOne uses mergeable branches to help train its models, then folds them before inference.
With its scaling strategy, MobileOne reaches strong accuracy while remaining efficient on the reported phone and desktop-CPU benchmarks. In the paper's iPhone 12 setting, S0 and S1 run under 1 ms; this does not apply to every variant or device.
The results improve the reported accuracy–latency trade-off and offer a useful starting point for future work and applications. The effect on generalization and speed still needs to be checked on the intended data and hardware.