Back to paper library
arXiv 2016CVPR 2017Object detectionPaper notes

YOLOv2

Expanding a Large Number of Categories

Original paper

YOLO9000: Better, Faster, Stronger

Paper authors

Joseph Redmon · Ali Farhadi

Publication
CVPR 2017Conference
First submitted to arXiv
arXiv:1612.08242
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{objectdetectionyolov22016,
  title = {{YOLO9000: Better, Faster, Stronger}},
  author = {Joseph Redmon and Ali Farhadi},
  year = {2016},
  eprint = {1612.08242},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1612.08242}
}
On this page

Problem Definition

Redmon et al., You Only Look Once, Figure 4; this is a YOLOv1 result

Looking back at YOLOv1, the authors noted many localization errors and relatively low recall in its final analysis.

Introducing anchor boxes seemed an obvious way to address those problems. Several better training methods had also appeared, and the authors considered them here as well.

Solution

Learn Box Priors and Constrain Centers to Cells

Redmon and Farhadi, YOLO9000, Figure 2

First, the anchor boxes. Earlier methods used fixed aspect ratios such as 1:1, 1:2 and 2:1. But if the model is meant for a particular dataset, why not find suitable anchors from that dataset?

The authors run K-means on the widths and heights of the annotated training boxes, using 1−IoU1-\mathrm{IoU} as the distance, to find suitable anchor dimensions. In the figure, they test different numbers of anchors on COCO and VOC. Five offers a useful trade-off between recall and model complexity in these experiments.

Redmon and Farhadi, YOLO9000, Figure 3

Faster R-CNN's RPN predicts offsets to adjust proposal boxes. YOLOv1 instead used a 7×77\times7 grid with two boxes per cell and a less constrained prediction space. YOLOv2 also predicts offsets, using anchor priors to make that job easier.

In the figure, the network predicts tx,ty,tw,tht_x,t_y,t_w,t_h rather than directly predicting x,y,w,hx,y,w,h:

bx=σ(tx)+cx,by=σ(ty)+cy,bw=pwetw,bh=pheth.b_x=\sigma(t_x)+c_x,\quad b_y=\sigma(t_y)+c_y,\quad b_w=p_we^{t_w},\quad b_h=p_he^{t_h}.

The sigmoid keeps the predicted center offset between 0 and 1, so the center stays within its grid cell; cx,cyc_x,c_y locate that cell. Exponentiating tw,tht_w,t_h gives positive width and height multipliers for the anchor dimensions pw,php_w,p_h.

When introducing anchors, the authors also changed the input from 448×448448\times448 to 416×416416\times416. The resulting 13×1313\times13 grid has a single central cell for large objects often near the image center. In the ablation, anchors alone slightly lower mAP from 69.5 to 69.2, but raise recall from 81% to 88%. Clustered dimensions and constrained center prediction are added later.

Fine Features and Variable Input

The authors also introduce multi-scale training. Every 10 batches, they randomly choose an input size in multiples of 32, from 320×320320\times320 through 608×608608\times608. This lets the same model work at different resolutions, with a speed and accuracy trade-off.

Darknet-19 and YOLO9000

Redmon and Farhadi, YOLO9000, Table 6

For faster inference, the authors designed their own backbone, Darknet-19. Like VGG, it mainly uses 3×33\times3 convolutions and increases channel count after pooling. It also uses batch normalization, 1×11\times1 convolutions and global average pooling for classification. After ImageNet pretraining and fine-tuning at 448×448448\times448, it reaches 76.5% top-1 and 93.3% top-5 accuracy. Training includes standard image augmentation, such as random crops, rotations and color changes.

For detection, the classification head is replaced with detection convolutions. A passthrough connection brings in higher-resolution features before the detector is fine-tuned.

Redmon and Farhadi, YOLO9000, Figure 6

To expand the number of detectable categories, the authors introduce WordTree. They train jointly on datasets with detection labels, which teach boxes and object presence, and classification datasets, which add many class names without box annotations.

How do we combine their labels? Detection datasets may say “dog” or “boat,” while classification datasets distinguish particular dog breeds. A single softmax over all these labels would treat a dog and its breed as mutually exclusive. WordTree uses WordNet to build a hierarchy, predicts conditional probabilities among children of each parent, and multiplies them along a path to a category. To estimate the probability of “Norfolk terrier,” for example, follow the path from the root to that breed and multiply the conditional probabilities.

With COCO detection data and ImageNet classification data, the authors train YOLO9000 to predict more than 9,000 classes in real time. On the ImageNet detection validation set, it reaches 16.0 mAP on the 156 classes without COCO detection annotations; this is a result for those classes in that evaluation, not equal detection quality across the whole vocabulary.

Experimental Results and Discussion

VOC Speed and Ablations

Redmon and Farhadi, YOLO9000, Table 3

On PASCAL VOC 2007, YOLOv2 offers a useful balance of speed and accuracy. At 288×288288\times288, it runs above 90 FPS, with mAP close to Fast R-CNN. That makes it attractive for smaller GPUs, high-frame-rate video or multiple streams. At 416×416416\times416, it reaches 76.8 mAP at 67 FPS, among the leading detectors of its time.

Redmon and Farhadi, YOLO9000, Figure 4
Redmon and Farhadi, YOLO9000, Table 2

The last table shows how the authors build YOLOv2 step by step: BatchNorm, high-resolution classification, a fully convolutional detection head, Darknet-19, clustered anchor dimensions, constrained center prediction, passthrough features and multi-scale training. Together these changes improve its speed and accuracy balance.

Notice that anchors alone reduce mAP from 69.5 to 69.2. After clustered dimensions and constrained center prediction are added, it rises to 74.4. Each table column reports the cumulative result, so these numbers should not be credited to anchors alone.

Conclusion

YOLOv2 adds anchors, BatchNorm and other training changes while keeping end-to-end detection fast. Its design improves recall and offers a range of speed and accuracy settings.

Its accuracy may be a little lower at the fastest settings, but that's all right: at least it's fast!

That speed helps explain why the model drew so much attention at the time.