Back to paper library
arXiv 2019CVPR 2020Efficient modelsPaper notes

GhostNet

Ghost in Feature Maps

Original paper

GhostNet: More Features from Cheap Operations

Paper authors

Kai Han · Yunhe Wang · Qi Tian · Jianyuan Guo · Chunjing Xu · Chang Xu

Publication
CVPR 2020Conference
First submitted to arXiv
arXiv:1911.11907
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{lightweightghostnet2019,
  title = {{GhostNet: More Features from Cheap Operations}},
  author = {Kai Han and Yunhe Wang and Qi Tian and Jianyuan Guo and Chunjing Xu and Chang Xu},
  year = {2019},
  eprint = {1911.11907},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.11907}
}
On this page

After the introduction of MobileNet, many researchers continued to work on improving the performance of lightweight networks.

This paper specifically addresses the problem of "feature map redundancy."

Defining the Problem

Han et al., GhostNet, arXiv:1911.11907v2, Figure 1

Convolutional networks often produce similar feature maps in different channels. These maps still carry useful information, but making each one with a full convolution costs computation. The figure highlights pairs of similar maps across channels: could some be generated from others with cheaper operations?

Solving the Problem

Generally, there are two common approaches: model compression and making the model more compact.

Model Architecture

Han et al., arXiv:1911.11907v2, Figure 2

In GhostNet, the authors proposed a new model architecture, as shown above.

Unlike typical convolutional networks, the GhostNet module operates in two steps:

  1. First, fewer convolutional filters generate a small set of intrinsic feature maps A. There are fewer intrinsic channels than the target output channels, though not necessarily fewer than the input channels.
  2. Then, linear operations are applied to each channel of the base feature map A to generate "ghost feature maps" B.

Finally, the base feature map A and the ghost feature maps B are concatenated to obtain the final feature map.

These channel-wise operations can be depthwise convolutions. Compared with full convolutions that mix input channels, they generate the additional maps at lower cost.

Module Configuration

Han et al., arXiv:1911.11907v2, Figure 3
  • Ghost Bottlenecks

    Leveraging the advantages of the ghost module, the authors introduced ghost bottlenecks (G-bnecks) designed specifically for small CNNs. This design is similar to the basic residual blocks in ResNet, combining multiple convolution layers and shortcuts.

    The ghost bottleneck primarily consists of two stacked ghost modules. The first ghost module acts as an expansion layer, increasing the number of channels, with the expansion ratio defined as the ratio of the output channels to the input channels. The second ghost module reduces the number of channels to match the shortcut path and connects the input and output of these two ghost modules through a shortcut.

    The first Ghost module is followed by batch normalization (BN) and ReLU. Following MobileNetV2, the second is followed by BN but no ReLU. For stride 2, a depthwise downsampling convolution is inserted between the modules, and the shortcut is resized to match.

  • GhostNet

    Based on ghost bottlenecks, the authors proposed GhostNet.

    This architecture generally follows the design of MobileNetV3, replacing its bottleneck blocks with ghost bottlenecks. GhostNet mainly consists of a series of ghost bottlenecks, starting with a standard convolution layer with 16 filters, followed by a series of ghost bottlenecks with gradually increasing channels. These ghost bottlenecks are divided into different stages based on the size of the input feature maps.

    Stride 2 is used where the network reduces spatial resolution; the other bottlenecks use stride 1. The configuration table below specifies these transitions. Finally, the feature map is converted to a 1280-dimensional feature vector for final classification through global average pooling and a convolution layer.

    Some ghost bottlenecks also use squeeze-and-excitation (SE) modules. Unlike MobileNetV3, GhostNet does not use hard-swish because of its latency cost in the authors' setting.

  • Width Multiplier:

    Although the basic architecture of GhostNet already provides low latency and high accuracy, in some cases, smaller, faster models or higher accuracy for specific tasks may be required.

    To meet different needs, the number of channels in each layer can be multiplied by a factor α\alpha, known as the width multiplier. This changes the width of the network, with model size and computation scaling roughly with α2\alpha^2. Channel-wise work and fixed-width layers mean this is only an approximation; actual latency still needs measurement.

    Generally, smaller α\alpha leads to lower latency and performance, while larger α\alpha improves performance. This design flexibility allows GhostNet to adapt to different application scenarios and requirements.

The final model configuration is shown in the table below:

Han et al., arXiv:1911.11907v2, Table 1

Discussion

Performance on ImageNet

Han et al., arXiv:1911.11907v2, Table 7

To verify the superiority of the proposed GhostNet, the authors conducted experiments on the ImageNet image classification task.

The authors follow earlier training settings, except that the initial learning rate is 0.4 for a batch of 1024 on 8 GPUs. All results report single-crop top-1 accuracy on the ImageNet validation set. For GhostNet, the primary convolution has kernel size k = 1, and all Ghost modules use s = 2 (two outputs per intrinsic channel, including the original) and d = 3 (the cheap convolution's kernel size).

The authors selected several modern small network architectures as competitors, including the MobileNet series, ShuffleNet series, ProxylessNAS, FBNet, and MnasNet, with results summarized in the table above.

These models are grouped around 50, 150, and 200–300 MFLOPs, common budgets for mobile applications. GhostNet is competitive in all three groups. For example, GhostNet 0.5× reports 42 MFLOPs/66.2% versus MobileNetV3 Small 0.75× at 44/65.4%, and GhostNet 1.0× reports 141/73.9% versus MobileNetV3 Large 0.75× at 155/73.3%. GhostNet 1.3× reaches 226/75.7% versus MobileNetV3 Large 1.0× at 219/75.2%: it is more accurate there, but also uses slightly more FLOPs.

Actual Inference Speed

Since the proposed GhostNet is designed for mobile applications, the authors further measured the actual inference speed of GhostNet on an ARM-based mobile phone using TFLite tools. In single-thread mode with a batch size of 1, they used common settings from the literature.

From the results below, it can be seen that GhostNet achieves approximately 0.5 percentage points higher top-1 accuracy than MobileNetV3 at the same latency, and GhostNet requires less runtime to achieve similar performance. For instance, GhostNet has only 40ms latency at 75.0% accuracy, whereas MobileNetV3 requires about 45ms to process an image at a similar accuracy level.

Han et al., arXiv:1911.11907v2, Figures 6–7

Conclusion

To reduce the computational cost of deep networks, the paper proposes a Ghost module for building efficient architectures. It is a convenient building block that can replace convolutions when designing a compact model, while retaining comparable performance after training. It is not an exact conversion that guarantees the original accuracy without retraining.

In the process of developing neural networks, the GhostNet architecture can be a viable option, especially in resource-constrained scenarios, potentially offering more possibilities.