Back to paper library
arXiv 2024NeurIPS 2024Object detectionPaper notes

YOLOv10

Goodbye NMS

Original paper

YOLOv10: Real-Time End-to-End Object Detection

Paper authors

Ao Wang · Hui Chen · Lihao Liu …

Show all 7 authors

Ao Wang · Hui Chen · Lihao Liu · Kai Chen · Zijia Lin · Jungong Han · Guiguang Ding

Publication
NeurIPS 2024Conference
First submitted to arXiv
arXiv:2405.14458
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{objectdetectionyolov102024,
  title = {{YOLOv10: Real-Time End-to-End Object Detection}},
  author = {Ao Wang and Hui Chen and Lihao Liu and Kai Chen and Zijia Lin and Jungong Han and Guiguang Ding},
  year = {2024},
  eprint = {2405.14458},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.14458}
}
On this page

We have followed the evolution from YOLOv1 through YOLOv9.

Now with YOLOv10, we can finally break free from the constraints of NMS—much to celebrate.

Problem Definition

In computer vision, real-time object detection plays an indispensable role.

From autonomous driving, industrial robots, AR/VR, to real-time analysis systems deployed at the edge, nearly all applications demand achieving high-accuracy classification and localization under extremely low latency. To meet these demands, the research community has continuously innovated, producing a proliferation of efficient detectors.

Since YOLOv1, one-stage detection has made it possible to obtain boxes and classes in one network pass. Later versions developed the familiar backbone, neck, and head division.

Subsequent versions like YOLOv3, YOLOv4 up to YOLOv9 have continuously optimized feature extraction and fusion capabilities by introducing modules such as CSPNet, PAN, ELAN, and GD. New techniques like YOLOv6’s SimCSPSPPF, YOLOv7’s E-ELAN, and YOLOv9’s GELAN + PGI attempt to push accuracy limits while maintaining speed.

However, regardless of innovation, most YOLO models remain constrained by the Non-Maximum Suppression (NMS) mechanism in the post-processing stage.

This limitation stems from the training phase’s one-to-many annotation strategy, which forces reliance on NMS during inference to select the best candidate boxes. This not only increases latency but also introduces hyperparameter sensitivity, hindering truly end-to-end inference.

To escape NMS limitations, End-to-End architectures led by DETR gained popularity.

DETR employs Transformers and Hungarian Matching to complete one-to-one label assignment, eliminating dependence on post-processing. Variants like Deformable-DETR, DINO, and RT-DETR have gradually improved convergence speed and operational efficiency. RT-DETR became one representative real-time DETR design, offering a promising path.

But can these Transformer-based designs truly replace CNNs?

In reality, although DETR architectures omit NMS, some DETR models have higher forward latency in the paper’s T4 comparisons; edge-device deployment depends on the hardware. Meanwhile, some studies have attempted to enable CNNs to support one-to-one correspondence directly—e.g., OneNet, DeFCN, FCOSpss propose various sample selectors or matching mechanisms aiming for end-to-end detection, but results remain unsatisfactory.

So, is it possible to transcend this binary choice?

Could we rethink the two key points: "training label assignment" and "architectural efficiency bottlenecks"? Or can we design a unified, lightweight, fast, and accurate new architecture that retains CNN advantages?

This paper’s authors attempt to fundamentally resolve the challenges faced by the YOLO series in real-time object detection.

Solution

YOLOv10’s technical core can be divided into two main threads:

  • First, freeing YOLO from NMS constraints to achieve true end-to-end inference;
  • Second, reassessing efficiency and accuracy of each network component, restructuring computational distribution from the architecture itself.

Dual Annotation Architecture

As shown above, YOLOv10 retains YOLO's backbone, neck, and detection-head design, adding a one-to-one head alongside the original one-to-many head. The two heads share the backbone and neck, and have the same structure and training objectives; they differ in how samples are matched.

During training, the one-to-many head uses TAL (Task-Aligned Label Assignment) for conventional multi-sample annotation, assigning multiple positive samples per ground truth to increase supervisory density; the one-to-one head pairs only one predicted box per ground truth, learning more precise sample correspondence.

This design allows the model to enjoy both dense and concise supervisory signals during training, ultimately keeping only the one-to-one head at inference while discarding the one-to-many branch, achieving truly "NMS-free" output.

However, the dual annotation also brings a potential issue: if the two heads use different sample matching logic, the supervisory signals learned will diverge, causing inconsistent training directions.

To solve this, the authors propose a simple and effective solution:

  • Consistent Matching Metric

The branches chiefly differ in how many positives each target receives. A different ranking metric can widen their supervision gap.

YOLOv10 unifies this score as follows:

m(α,β)=s⋅pα⋅IoUβm(\alpha, \beta) = s \cdot p^\alpha \cdot \text{IoU}^\beta

Where:

  • pp is the predicted classification score;
  • IoU\text{IoU} is the Intersection-over-Union between predicted box and ground truth;
  • ss is the spatial prior, indicating whether the prediction point lies within the ground truth;
  • α\alpha and β\beta are hyperparameters adjusting the weights of classification and localization information.

If the heads use α\alpha, β\beta values that are not proportional, candidates may receive different rankings in each branch’s view, causing “inconsistent supervision direction.”

The one-to-one α,β\alpha,\beta are set to the same positive multiple rr of the one-to-many values, preserving the ranking of candidates. The default is r=1r=1, with both heads using α=0.5\alpha=0.5, β=6\beta=6. Their full classification targets still differ.

The next question is:

Does this adjustment really align the two heads?

Figure (b) above provides experimental evidence.

It shows whether samples selected by the one-to-one head in a YOLOv8-S model also rank in the one-to-many head’s Top-1, Top-5, and Top-10 selections.

The contrasting colors represent different settings:

  • Blue: “Inconsistent matching” setting with α=0.5\alpha = 0.5, β=2\beta = 2;
  • Orange: “Consistent matching” setting with one-to-one α=0.5\alpha = 0.5, β=6\beta = 6. The one-to-many head uses α=0.5\alpha = 0.5, β=6\beta = 6 in both settings.

Under consistent settings, a greater share of the samples selected by the one-to-one head rank within the one-to-many head's Top-1, Top-5, and Top-10. This supports improved supervision alignment, although Figure 2(b) does not directly measure feature quality.

Efficiency and Accuracy Optimization

Although past YOLO series have undergone multiple module innovations, they still lack a systematic method to evaluate the overall architectural design logic. Therefore, the authors approach the problem from a global perspective, attempting to optimize both efficiency and accuracy simultaneously, so that each computational unit’s cost is fully leveraged.

The YOLO architecture can be primarily divided into four parts:

  • Stem (initial convolution layers);
  • Downsampling layers;
  • Stage module stacks;
  • Detection head.

Considering the stem’s minimal computational cost, the authors focus optimization efforts on the latter three.

  1. Classification and Regression Heads

    In YOLO, classification and regression heads typically share the same design, but detailed analysis reveals a large cost difference. For example, in YOLOv8-S, the classification head requires 5.95G FLOPs and has 1.51M parameters—2.5× the computation and 2.4× the parameters of the regression head.

    In the authors' error-replacement analysis, correcting regression errors yields a larger AP gain than correcting classification errors. Therefore, they redesign the classification head into two 3×3 depthwise-separable convolutions followed by a 1×1 convolution, significantly reducing computational load while retaining basic classification capability.

  2. Spatial and Channel-Decoupled Downsampling

    The conventional downsampling uses a stride=2 3×3 convolution, which simultaneously reduces spatial dimensions and increases channel count.

    Although seemingly simple, this incurs substantial cost, with computation complexity:

    O(92HWC2)\mathcal{O}\left(\tfrac{9}{2}HWC^2\right)

    and parameter count:

    O(18C2)\mathcal{O}(18C^2)

    The authors adopt a separated design: first using a pointwise (1×1) convolution to adjust channels, then a depthwise (3×3) convolution to perform downsampling.

    This modification reduces computational complexity to:

    O(2HWC2+92HWC)\mathcal{O}\left(2HWC^2 + \tfrac{9}{2}HWC\right)

    and parameters to:

    O(2C2+18C)\mathcal{O}(2C^2 + 18C)

    while preserving more feature information, enabling faster forward passes without compromising accuracy.

  • Rank-Guided Block Design

    YOLOv10 uses a measurable proxy to inspect redundancy across stages.

    The authors calculate the numerical rank of the weight matrix in the last convolution layer of each stage, normalizing it by rCout\tfrac{r}{C_\text{out}}, where CoutC_\text{out} is the output channel count, and use half of the largest singular value λmax\lambda_{\text{max}} as the threshold.

    Figure (a) shows lower normalized rank in some deeper stages and larger models; rank does not decline monotonically across stages.

    Here, lower rank is a redundancy proxy, suggesting that later stages may have room for compression, reinforcing the intuition that a one-size-fits-all structure is suboptimal.

    Therefore, the authors introduce a new module design: Compact Inverted Block (CIB), illustrated in figure (b), which uses depthwise convolution for spatial mixing and pointwise convolution for channel fusion, significantly reducing computation cost.

    They then adopt a rank-guided iterative replacement strategy: starting from the stage with the lowest rank, replace modules one by one with CIB; if performance remains stable, continue replacing; otherwise, stop. This allows the model to automatically adjust depth and structure according to scale, seeking an efficiency-accuracy trade-off for that model and validation setting.

Accuracy-Oriented Enhancements

Beyond simplification, YOLOv10 also strengthens high-level semantic modeling and recognition over large receptive fields.

  • Large Convolution Kernels: Enlarging Receptive Field Only Where Needed

    Large kernels can enlarge the receptive field, but applying them throughout the model may interfere with shallow features used to detect small objects and increase I/O and latency at high resolutions.

    Therefore, the authors apply large 7×7 depthwise kernels only in the second depthwise convolution of deep CIB blocks, paired with reparameterization techniques adding a 3×3 branch to stabilize optimization. During inference, this branch is fused back into the main path without extra cost.

    Note that large kernels are used only in smaller models (e.g., YOLOv10-S/N), since as models scale up, the intrinsic receptive field grows and marginal benefits of large kernels diminish.

  • PSA Module: Not All Attention Is Worth Paying

    Although self-attention offers powerful global modeling, with H×WH\times W spatial tokens, its attention term grows quadratically with token count, roughly O((HW)2)\mathcal{O}((HW)^2) is unfriendly for real-time models. To address this, the authors design a Partial Self-Attention (PSA) module.

    This module splits feature channels into two halves: one half passes through multi-head self-attention (MHSA) and feed-forward network (FFN), while the other half remains unchanged; both are concatenated and fused via a 1×1 convolution. To further improve speed, they employ:

    • Half-dimensional Q/K projections;
    • BatchNorm replacing LayerNorm;
    • Insertion only after Stage 4, where resolution is lowest.

    This selective self-attention design enables YOLOv10 to introduce global context at minimal cost, significantly improving classification accuracy and semantic consistency.

Implementation Details

This work takes YOLOv8 as the main baseline model for two reasons: it currently offers a good balance between speed and accuracy, serving as a representative comparison; and it publicly supports multiple model sizes, facilitating fair benchmarks.

YOLOv10 inherits YOLOv8’s size divisions—N / S / M / L / X variants—and additionally introduces YOLOv10-B, a width-expanded version of YOLOv10-M designed to fill the gap between mid-tier model accuracy and capacity.

All models are trained and evaluated on the COCO dataset, following the same training-from-scratch settings as YOLOv8 and YOLOv9.

Table 1 measures latency on NVIDIA T4 with TensorRT FP16. It distinguishes end-to-end latency including post-processing from forward-only LatencyfLatency^f; comparisons still use each model’s official weights and implementation.

All YOLOv10 models in this study adopt the consistent dual-annotation training strategy and structural optimizations, rather than single-point changes.

Discussion

Comparison with SOTA Methods

YOLOv10 demonstrates significant advantages across multiple model scales.

As shown in the table above, compared to the original YOLOv8, across five common sizes (N / S / M / L / X):

  • AP increases by 0.3–1.4 points;
  • Parameters fall by 28–57%;
  • FLOPs fall by 23–38%;
  • End-to-end T4 latency falls by up to 70% in the table.

In these five YOLOv8 comparisons, YOLOv10 improves AP, parameter count, FLOPs, and end-to-end latency. The stepwise ablations below are needed to separate the contributions of its individual designs.

In cross-model comparisons:

  • YOLOv10-N / S improve on YOLOv6-3.0-N / S by 1.5 / 2.0 AP points, with 51% / 61% fewer parameters and 41% / 52% fewer FLOPs. The N FLOP reduction is below 50%;
  • YOLOv10-B / M compared to YOLOv9-C and YOLO-MS, maintain or improve accuracy while reducing latency by 46% / 62%, making them particularly suitable for latency-sensitive deployments;
  • YOLOv10-L is smaller than Gold-YOLO-L (68% fewer parameters, 32% lower latency) yet scores 1.4 AP points higher.

These selected comparisons show YOLOv10's advantages in speed and resource use. The table also has a counterexample: YOLOv9-S has higher AP than YOLOv10-S, so not every metric favors YOLOv10 for every model.

When compared to non-YOLO real-time detectors, YOLOv10 remains competitive. For instance, against the well-known end-to-end RT-DETR architecture:

  • YOLOv10-S is 1.8× faster than RT-DETR-R18;
  • YOLOv10-X is 1.3× faster than RT-DETR-R101;
  • Table 1 reports nearby AP for these pairs: 46.3 versus 46.5, and 54.4 versus 54.3, respectively.

Ablation Study

To examine the effects as the designs are added, the authors conduct ablations on YOLOv10-S and YOLOv10-M, as shown above. The experiments sequentially add three core designs and track latency, parameter count, computation, and accuracy.

First, on YOLOv10-S, simply introducing the NMS-free training mechanism (i.e., the consistent dual-annotation strategy) reduces end-to-end latency from 7.07 to 2.44 ms, but AP falls from 44.9 to 44.3: a 4.63 ms gain for 0.6 AP point.

Next, by further applying the efficiency-oriented architectural optimizations on the YOLOv10-M NMS-free baseline, the model cuts 11.8M parameters and 20.8 GFLOPs, with inference latency dropping by 0.65 ms. This shows that designs like the Rank-Guided Block and decoupled downsampling effectively reduce computational redundancy.

Finally, after the accuracy-oriented step, AP rises by 1.8 points on YOLOv10-S and 0.7 points on YOLOv10-M, while latency rises by 0.18 ms / 0.17 ms. M does not use large-kernel convolutions, so its gain at this step comes from PSA; this sequential table does not isolate the two modules' contributions.

Can NMS-free Truly Replace Traditional Methods?

Although YOLOv10 demonstrates excellent overall performance and very low latency under NMS-free training, the authors honestly point out that compared to traditional training combined with NMS, there remains a certain accuracy gap.

However, this gap is not fixed.

Observations show that the larger the model, the smaller the AP difference between the two training strategies. For YOLOv10-X, both score 54.4 AP; for the smaller YOLOv10-N, NMS-free training AP is 1.0 point lower than traditional one-to-many training with NMS.

Where does this gap come from?

The authors suggest that one-to-one assignment relies more heavily on discriminative features. Since each ground truth corresponds to only one predicted sample, the capacity of smaller models may make it harder to select an ideal match.

To further verify this hypothesis, the authors plotted the following:

Analyzing the average cosine similarity between each anchor’s feature and those of other anchors on the COCO validation set.

Mean similarity decreases as model size increases. The authors interpret this as more discriminative features that may help one-to-one matching.

This also leads to a reasonable inference:

In this experiment, smaller models may have more difficulty learning separated features for one-to-one training. The AP gap is smaller for large models.

Why Doesn’t Simplification Drag Performance?

YOLOv10’s efficiency-oriented design includes three key adjustments: a lightweight classification head, spatial-channel decoupled downsampling, and rank-guided module allocation.

Experimental results show these changes reduce parameters and computations without sacrificing accuracy.

Specifically, simplifying the classification head changes S AP from 44.3 to 44.2; the error analysis shows a larger gain from correcting regression errors. The 0.7-point AP gain for downsampling is against the alternative DW→PW order in the appendix, not one step of the cumulative table above. CIB and rank-guided replacement do not reduce AP when replacing Stage 8 and Stage 4, but adding Stage 7 lowers AP from 44.5 to 44.3.

Accuracy Improvements with Nearly No Latency Increase?

Following efficiency optimizations, YOLOv10 introduces two accuracy-oriented designs: large-kernel convolutions and Partial Self-Attention (PSA). These modules enhance the model’s receptive field and semantic modeling capacity but add minimal latency, especially effective for small models.

For example, on YOLOv10-S, adding 7×7 large-kernel convolution and PSA modules raises AP by 0.4 and 1.4 points, respectively, while latency increases by only 0.03 ms and 0.15 ms.

Since large models already have sufficiently large receptive fields, the benefits of large kernels are limited and thus only applied to smaller models (e.g., YOLOv10-N / S).

In this YOLOv10-S ablation, PSA is 0.05 ms faster than the compared full Transformer block and scores 0.3 AP point higher. The authors suggest that attending to only some channels may reduce redundancy and optimization difficulty. Increasing the number of PSA blocks from one to two adds 0.2 AP point at another 0.10 ms, so the default is one.

Conclusion

YOLOv10, through its NMS‑free training process and overall architectural simplifications, delivers a cleaner inference pipeline and a superior speed-accuracy trade-off for real-time object detection.

For developers, the greatest value lies in deployment simplification: removing the NMS step and concentrating operator types makes the model easier to maintain and optimize in mainstream inference frameworks like TensorRT and ONNX Runtime.

Before deployment, two engineering questions remain:

  • The paper reports FLOPs and T4/TensorRT FP16 latency; multi-stream cloud-GPU and Jetson Orin performance still needs measurement;
  • On NPUs and mobile devices, latency will depend on support for depthwise, large-kernel, and PSA operators. This paper does not benchmark those devices against YOLOv8/9.

Currently, NMS‑free training still has a 1.0-point AP gap for N against the same architecture trained one-to-many with NMS, indicating that improving small model feature discriminability remains a key future optimization focus.

Overall, considering deployment platform support for operators, remember to test performance before adopting!