Back to paper library
arXiv 2018Object detectionPaper notes

YOLOv3

Introducing Multi-Scale Detection

Original paper

YOLOv3: An Incremental Improvement

Paper authors

Joseph Redmon · Ali Farhadi

Publication
Publication venue not yet verified
First submitted to arXiv
arXiv:1804.02767
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{objectdetectionyolov32018,
  title = {{YOLOv3: An Incremental Improvement}},
  author = {Joseph Redmon and Ali Farhadi},
  year = {2018},
  eprint = {1804.02767},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.02767}
}
On this page

The third generation of YOLO is not a formal research paper. As the author himself puts it, this is a technical report.

Problem Definition

There have been many recent advances in object detection.

The author thought it was time to bring some of these ideas into YOLO and make a few improvements.

The bounding-box prediction design remains the same as YOLOv2, so we will not go through it again here.

Solution

The author made a series of improvements.

Darknet-53: Deeper, With Measured Throughput

Redmon and Farhadi, YOLOv3: An Incremental Improvement, Table 1

The author replaced Darknet-19 with Darknet-53. It still uses successive 3×3 and 1×1 convolutional layers, now with residual connections. This deeper network was trained on ImageNet; its 53 convolutional layers give Darknet-53 its name.

The table below shows Darknet-53 on ImageNet. Its top-1 accuracy is 77.2%, close to ResNet-101's 77.1%. Under the table's 256×256, Titan X test conditions, it runs at 78 FPS versus ResNet-101's 53 FPS, or about 1.5 times as fast.

Redmon and Farhadi, Table 2; 256×256 classification on Titan X

Three Scales and Independent Class Scores

Instead of softmax, YOLOv3 uses logistic regression to predict the probability of each class.

This does not force each box to belong to only one class. The same box may have overlapping labels such as “woman” and “person” in the dataset.

Multi-Scale Prediction

Drawing on ideas from FPN and RetinaNet, the author introduced multi-scale detection in YOLOv3.

K-means clustering still determines the anchor-box dimensions. The difference is that YOLOv3 predicts from feature maps at three scales: 1/32, 1/16, and 1/8 of the input. Deeper features are upsampled and concatenated with shallower features before prediction at the next scale. In the COCO experiment, each location at each scale predicts three boxes. Every box has four bounding-box offsets, one object-confidence score, and 80 class probabilities.

The prediction tensor at a given scale has shape N×N×[3×(4+1+80)]N \times N \times [3 \times (4 + 1 + 80)], where NN is that scale's feature-map width and height.

Experimental Results and Discussion

COCO: Separate AP From AP50

Redmon and Farhadi, Figure 3; comparison plot adapted from RetinaNet
Redmon and Farhadi, Table 3; comparison rows from RetinaNet

The author openly admits that, rather than spend a great deal of time retraining other architectures, he “borrowed” a plot and a table from the RetinaNet paper and added YOLOv3's results. The comparisons show a good balance of speed and accuracy.

In the table, YOLOv3 does particularly well on AP50AP_{50}, far above the SSD architecture shown there. Its performance falls as the IoU threshold rises, however, suggesting that it has more difficulty producing tightly localized boxes.

Earlier YOLO models struggled with small objects, but YOLOv3 reverses that trend. With the new multi-scale detector, small-object AP rises from YOLOv2's 5.0 to 18.3. Medium- and large-object AP also improve over YOLOv2, although they still trail some of the other models in the table.

Failed Experiments Are Useful Too

The author also reports several training experiments that did not help:

  1. Anchor-center x, y offsets: Predicting offsets in units of box width and height with a linear activation made the model less stable and performed poorly.
  2. Linear x, y prediction instead of logistic: Directly predicting the offsets with a linear activation reduced mAP by a few points.
  3. Focal Loss: This attempt to address class imbalance reduced mAP by about two points. The author says he does not know why.
  4. Two IoU thresholds: Following Faster R-CNN, treating IoU above 0.7 as positive, below 0.3 as negative, and ignoring the middle range did not work well here.

Conclusion

Previous research suggests that people have trouble telling detection boxes with IoU 0.3 and 0.5 apart by eye. The author therefore questions how meaningful it is to keep chasing higher scores on these validation datasets.

In his view, YOLOv3 already performs well enough for most real-world problems. We should focus on model safety and on using these powerful models responsibly, so that they cause less harm in the world.