FCOS: Fully Convolutional One-Stage Object Detection
- Publication
- ICCV 2019Conference
- First submitted to arXiv
- arXiv:1904.01355
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{objectdetectionfcos2019,
title = {{FCOS: Fully Convolutional One-Stage Object Detection}},
author = {Zhi Tian and Chunhua Shen and Hao Chen and Tong He},
year = {2019},
eprint = {1904.01355},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1904.01355}
}
At the time of the author's note, this paper had already drawn more than seven thousand citations.
A classic paper that we must take a look at.
Problem Definition
For a long time, mainstream object detection methods have been dominated by anchor-based designs.
From Faster R-CNN, SSD to YOLOv2/v3, everyone has centered their designs around anchors: laying out a huge number of candidate boxes and then deciding which are positive and which are negative samples.
While practical, this design brings several concerns:
- Numerous hyperparameters: Anchor size, aspect ratio, count, and IoU thresholds need tuning. The FCOS paper cites a RetinaNet ablation where anchor configuration changes COCO AP by up to about four points.
- Poor generalization: Fixed anchor shapes can make objects of very different sizes or unusual shapes difficult to cover, and a new detection task may require redesigning them.
- Imbalance of positive and negative samples: To ensure recall, thousands of anchors must be tiled across the image. The vast majority become negative samples, leading to extremely imbalanced training.
- Complex computation: From IoU calculations to matching rules, anchors introduce additional computation and complexity.
Although anchor-free is not a new concept, previous methods each had obvious flaws.
For example, YOLOv1 predicts boxes from grid locations near object centers and has limited recall; CornerNet pairs detected corners but needs extra grouping work; DenseBox regresses from positions directly, yet varying object sizes and overlapping boxes made generic detection difficult. The latter family was mainly used for specialized tasks such as text or face detection.
There are many more examples.
Anchor-free architectures have always been a step behind in overall performance—in the field already ruled by anchor-based methods, they had no real foothold.
Does object detection really depend on anchors?
The authors aim to break this deadlock: like semantic segmentation, the detector could predict at each sampled feature-map location, without anchors or their IoU matching rules. FCOS still needs rules for assigning positive locations and scales.
Object detection, in their view, should return to the clean paradigm of fully convolutional per-pixel prediction.
Solution
The core idea of FCOS is to reframe object detection as a per-pixel prediction problem. Instead of matching a grid of candidate anchors with ground truth, it treats sampled feature-map locations mapped to the image as training samples.
Suppose at the -th layer of the backbone, we obtain a feature map with stride .
The ground-truth bounding box for an input image is denoted as
where and are the coordinates of the top-left and bottom-right corners, and is the class label.
For a location on the feature map, write its corresponding input-image coordinates as :
If this image position falls within a ground-truth box , it is a positive sample with class label ; otherwise, it is background ().
Besides classification, each location must regress a 4D vector:
which represents the distances from that position to the left, top, right, and bottom edges of the box, as illustrated below:
If a position falls within multiple boxes, the box with the smallest area is chosen as the regression target to reduce ambiguity.
In this way, each sampled location directly outputs “what is here” and “where is the box,” without relying on anchors.
At the network output, the last layer outputs both a classification branch and a regression branch .
Since regression distances must be positive, the authors use to map any real number to . When the regression head is shared across FPN levels, they use a learned scale for each level and output .
Compared with a RetinaNet-like detector using nine anchors at each location, this design needs roughly one ninth as many class and box output variables per location. That is a comparison with that nine-anchor setup, not a general ninefold speedup.
The overall loss is defined as:
Here uses Focal Loss across locations, while the indicator restricts IoU regression loss to positives. is the number of positive samples and . Once center-ness is added, its BCE loss is trained separately.
During inference, for each location , we obtain its classification score and regression result . As long as , it is treated as a candidate box and mapped back to image coordinates by inverse transformation.
At this point, the core idea of FCOS is clear: remove anchors and let each sampled location predict directly.
However, this design immediately faces two challenges:
- The stride of the feature map may be too large, causing small objects to be missed and recall to drop;
- Overlapping boxes cause ambiguity—which box should a pixel regress to?
The authors' solution is multi-level prediction via FPN, as shown below:
The authors use five levels () with strides of 8, 16, 32, 64, and 128.
The authors assign a positive location to a level according to the maximum of its four side distances to the ground-truth box: for example, covers the 0–64-pixel regression range and the 64–128-pixel range. This is not a direct threshold on box area. Objects of different sizes therefore often land on different levels, reducing overlap ambiguity.
Even so, there remains a final issue: pixels far from the center often produce low-quality boxes.
To address this, the authors introduce a lightweight center-ness branch to measure how far a location deviates from the object center.
For position , the target is defined as:
This value lies in , approaching 1 near the object center.
BCE loss supervises this branch during training. At inference, its score is multiplied by the classification score, lowering the rank of edge-location predictions so that poor boxes are more likely to be removed during subsequent NMS.
Ultimately, FCOS, with its simplest per-pixel design, together with multi-level assignment via FPN and centerness for quality control, completely abandons anchors—yet demonstrates performance comparable to, and even surpassing, mainstream anchor-based methods.
Discussion
The authors validate FCOS on the MS COCO benchmark, following standard practice: trainval35k (115K) is used for training, minival (5K) for ablation, and main results are submitted to test-dev (20K).
Unless otherwise specified, the backbone is ResNet-50, and the optimizer and hyperparameters follow the RetinaNet settings: SGD, 90K iterations, batch size 16, initial learning rate 0.01, which drops by a factor of 10 at both 60K and 80K iterations, weight decay 0.0001, momentum 0.9, input image short side 800, and long side no more than 1333.
Inference also adopts RetinaNet's post-processing and hyperparameters, with candidate location threshold ; further performance gains are possible if parameters are specifically tuned for FCOS.
Before the rest of the analysis, let us clarify Best Possible Recall (BPR) so that the tables are easier to read:
Multi-Level FPN and Recall Upper Bound
First, the authors consider that FCN-style detectors may suffer from insufficient recall due to larger stride, so they test with a single P4 (stride 16) layer: with only this layer, FCOS still achieves BPR = 95.55%, which is significantly higher than the official Detectron version of RetinaNet (low-quality matching, IoU≥0.4), which achieves 90.92%.
With FPN (P3–P7), FCOS reaches 98.40% BPR, 0.83 percentage points below RetinaNet's 99.23% when all low-quality matches are allowed.
BPR is assignment-based recall coverage, not the detector's measured recall. FCOS's AR is higher than RetinaNet's in the controlled comparison below, so these experiments do not show the BPR gap becoming the main performance bottleneck.
Ambiguous Samples Caused by Overlaps
When multiple GT boxes overlap in space, per-pixel assignment may cause ambiguity as to "which box a given location should regress to."
With only P4, ambiguous samples on minival account for 23.16% of all positive samples; after introducing FPN, because objects of different sizes are assigned to different levels, ambiguity drops sharply to 7.14%. Excluding "same-class overlaps" (which do not affect classification accuracy), the cross-class ambiguity rate drops from 17.84% to 3.75%.
At inference time, only 2.3% of detected boxes originate from ambiguous locations, and for cross-class ambiguities, this drops to 1.5%.
Thus multi-level assignment plus the smallest-area rule substantially reduce ambiguity. These rates and the subsequent AP results support the method.
The Necessity and Effectiveness of Center-ness
Even with FPN, pixels far from the object center still easily generate low-quality boxes.
To address this, FCOS adds a single-layer center-ness branch alongside the classification branch, predicting the normalized offset from center:
This is supervised with BCE during training. At inference, lowers edge predictions in the ranking, making poor candidates more likely to be filtered by subsequent NMS.
In this minival ablation, a predicted center-ness branch raises AP from 33.5% to 37.1%. Computing center-ness directly from regressed sides instead leaves AP at 33.5%. This supports a separate learned branch in this setup.
Fair Comparison With Anchor-based Methods
To rule out implementation differences, the authors adjust FCOS to settings nearly identical to RetinaNet: removing GroupNorm (using gradient clipping for stability instead), generating P6/P7 from C5 rather than P5.
In this minival comparison, FCOS has 36.3% AP versus RetinaNet's 35.9%, but its AP50 is 54.8% versus 56.0%. Learning rate, NMS threshold, and other hyperparameters follow RetinaNet. The result supports FCOS as an anchor-free alternative; it cannot by itself isolate the contribution of a simpler architecture or of assignment rules. The authors also suggest FCOS may improve with dedicated tuning.
Overall Comparison With Contemporary SOTA
On test-dev, FCOS with ResNet-101-FPN reaches 41.5 AP versus 39.1 AP for a same-backbone RetinaNet row, a 2.4-AP cross-method gap whose training conditions are not fully controlled. With a ResNeXt-101-64×4d-FPN backbone, FCOS reaches 43.2 AP, above anchor-free CornerNet's 40.5 AP, while avoiding its corner pairing and embedding aggregation.
With center-ness branch placement, center sampling, GIoU, and regression-target normalization added, the single-model, single-scale result reaches 44.7 AP. This improved system is a separate row from the 43.2-AP FCOS result; the extra changes should remain visible when comparing them.
Conclusion
FCOS brings object detection back to the simple per-pixel paradigm: no proposals, no predefined anchors. Each sampled location learns both “what is here” and “where is the box.” FPN assignment reduces size and overlap ambiguity, and a lightweight center-ness branch helps rank box quality.
The influence of FCOS is reflected on three main levels.
First, sample assignment and task alignment became active lines of work. ATSS studies statistical assignment; PAA, OTA, and YOLOX's SimOTA explore other dynamic or optimization-based assignments; TOOD works on classification-localization alignment. They tackle related questions about which locations should learn from which objects, although these works should not all be presented as direct descendants of FCOS.
Second, quality-aware scoring and box regression took several paths. GFL's QFL/DFL links classification scores to localization quality and uses distributional box regression. VarifocalNet uses IoU-aware scores and star-shaped box features. These are related ways to reflect box quality; their mechanisms are not interchangeable with FCOS center-ness or with each other.
Third, the Transformer line, starting with DETR and followed by Deformable DETR, DINO, and RT-DETR, offers another route using set matching that can avoid anchors and NMS. In real-time and edge settings, lightweight convolutional heads like FCOS remain useful baselines. Later anchor-free YOLO designs also address assignment and quality scoring.
In summary, FCOS's contribution is not just "removing anchors," but reframing the detection problem: let each location make its own decision, and use correct assignment and scoring so that good boxes naturally emerge.
Anchor-based methods did not disappear. FCOS left a clearer way to ask how locations are assigned and how good boxes are scored.