Back to Blog
Articles

Six Years After NMS Was Dropped, Why Did YOLO26 Bring It Back?

In 2020, DETR said it didn't need NMS; in September 2026, YOLO26's default put NMS back in. Starting from a dog boxed three times, we look at what one-to-one matching saved and what it cost training, then at how the DETR and YOLO lines met in the middle.

On September 5, 2026, Ultralytics released ultralytics 8.4.142. According to the release notes, YOLO26 prediction and validation now default back to the one-to-many output head followed by NMS, because it “generally provides higher accuracy than the NMS-free one-to-one head.” If you want the original path, you have to add nms=False yourself. Ultralytics v8.4.142

Eight months earlier, when the same model officially launched, the first highlight in its release notes said exactly the opposite. Its native end-to-end design needed no NMS post-processing at all, which meant lower latency and simpler export. Ultralytics v8.4.0

Go back another six years. In 2020, the abstract of Facebook AI's DETR said its approach could remove many hand-designed components “like a non-maximum suppression procedure or anchor generation.” DETR abstract

NMS was taken out, then put back in. This article starts with why a single object gets boxed several times, looks at how DETR gives each object just one prediction and what that cost, and then follows the DETR and YOLO lines as they end up meeting in the middle.

One Dog, Three Boxes

A detector never guesses just one box. In YOLO26, for example, a 640×640 input gives 8,400 positions across feature maps at three resolutions, and each position outputs a set of class scores and a box. YOLO26 docs

During training, the YOLO family uses one-to-many label assignment. Several positions near an annotated object, those whose scores and boxes are both good enough, all count as positives; YOLO26's one-to-many head takes at most 10 per object. YOLO26 paper §3.2.1 Taught this way, when the model sees a dog, several nearby positions all say “there's a dog here.”

So at inference, the same dog shows up in several boxes. This is where NMS (non-maximum suppression) cleans up. It sorts the boxes by score, keeps the highest one, deletes the boxes of the same class that overlap it too much, and then repeats on the boxes that remain. Overlap is measured with IoU, the area where two boxes intersect divided by the area of their union.

Let's try it with four boxes. A, B, and C all sit on the same dog, and D frames the cat next to it. A has the highest score; B's IoU with A is 0.904, and C's is 0.656. With the threshold at 0.5, B and C are both deleted, so the dog and the cat each keep one box. Loosen it to 0.7, and C escapes deletion, so the same dog gets an extra box.

Set the threshold too strict, on the other hand, and you run into a different problem. If C actually frames a second dog pressed right up against A, a 0.5 threshold will delete it as a duplicate. Crowd photos run into this most often. The CrowdHuman dataset averages 22.6 people per image, with about 2.4 pairs of person boxes per image overlapping at an IoU above 0.5; COCO's photos of people average only 0.02 pairs. The authors also write that lowering the threshold makes recall drop sharply, while raising it brings more false positives. CrowdHuman §1

The thresholds also tie accuracy and time together. The RT-DETR authors measured YOLOv8 on a T4 GPU with the IoU threshold fixed at 0.7; raising the confidence threshold from 0.001 to 0.05 lowered AP from 52.9 to 51.2, while the time spent in NMS itself (TensorRT's EfficientNMS) fell from 2.36 ms to 1.06 ms. RT-DETR Table 1 NMS is also a post-processing step that has to be deployed separately, and each inference platform the model is exported to has to handle it on its own.

The right half holds the IoU threshold fixed and varies confidence, showing how AP and NMS time change together.

AP and NMS time are separate columns. Time is in milliseconds for the EfficientNMS kernel on a T4 with TensorRT FP16. DETRs Beat YOLOs on Real-time Object Detection — Table 1, Zhao, Yian et al..

DETR Gives Each Object Just One Prediction

DETR sets up a fixed number of prediction slots (queries), 100 in the standard model. Each slot outputs one class and one box, and the classes include an extra “no object.”

During training, DETR doesn't let several slots claim the same object at once. It makes a one-to-one matching between predictions and annotations, choosing the assignment with the lowest total cost among all possible ones; the cost considers both whether the class score is right and how far off the box is. This optimal matching is found with the Hungarian algorithm. DETR §3.1

Let's walk through it with three slots and two objects. The table gives the cost of assigning each slot to each object; lower is better:

SlotDogCat
1 (on the dog)0.21.5
2 (also on the dog)0.31.4
3 (on the cat)1.60.4

The lowest-cost assignment pairs slot 1 with the dog and slot 3 with the cat, for a total cost of 0.6; slot 2 is left unmatched, and its training target is “no object.”

Slot 2's box isn't actually bad; it costs only a little more than slot 1's. But the dog already has a slot in charge of it, so slot 2 gets “no object” as its training target. Keep training this way, and the second box on the same object gets suppressed, so inference has no need for NMS.

The DETR authors also ran an NMS check on the output of every decoder layer. NMS helps after the first layer, because predictions from a single layer tend to repeat; by the later layers, the self-attention among slots has learned to keep them out of each other's way, and NMS ends up deleting correct boxes, lowering AP slightly. DETR §4.2, Figure 4

Solid and dashed lines of the same color compare direct predictions with NMS.

The x-axis counts decoder layers. Blue uses the left AP axis; orange uses the right AP50 axis. The early benefit of NMS fades in later layers. End-to-End Object Detection with Transformers — Figure 4, Carion, Nicolas et al., CC0 1.0.

To match Faster R-CNN's 42.0 AP, though, DETR had to train for 500 epochs. By the Deformable DETR authors' tally, that comes to about 2,000 GPU hours, against just 380 for Faster R-CNN. Deformable DETR Table 1 DETR also struggled more with small objects, with an AP_S of 20.5 against Faster R-CNN's 26.6. DETR Table 1

Three Hundred Predictions, Fewer Than Thirty Learning Boxes

Later papers gave different diagnoses for why DETR learned so slowly. Deformable DETR traced the problem to attention, which starts out spread evenly over the whole feature map and takes a long time to learn to focus. Deformable DETR §1 DN-DETR traced it to unstable matching; the same slot on the same image may be matched to the dog this round and to the cat the next. DN-DETR §1 A third explanation is that there are too few positives. This article follows that one, because it bears directly on NMS.

As the H-DETR authors point out, Deformable DETR has 300 slots per image, and nearly 99% of COCO photos have fewer than 30 annotated boxes, so fewer than 30 slots per image get matched to an object and learn where a box should go; the other 270-plus slots get only classification training toward “no object.” H-DETR §1

The DEIM authors plotted the number of positives directly. Under one-to-one matching, the count per image stays mostly within 10; switch to YOLO-style one-to-many assignment, and a single photo can have more than 80 positives, about 10 times as many in the extreme cases. DEIM §3.2, Figure 3

One-to-one inference is clean, with one box per object, but the supervision during training is too sparse; one-to-many supervision is dense, but inference then always needs NMS to clean up. Over the next six years, both lines kept looking for a way to have it both ways.

On the DETR Side, One-to-Many Returns in Training

In 2022, H-DETR kept the original 300 one-to-one slots and, during training, added another group of 1,500 slots, copying each annotated object 6 times and matching this group one-to-many; a mask separates the two groups so they don't interfere with each other. At inference, only the original 300 one-to-one slots remain. H-DETR §3.2

Start with Hybrid-Branch on the left: blue serves training and inference; green is used only for training.

The other columns switch matching by training epoch or decoder layer. They are distinct alternatives to the two-branch design on the left. DETRs with Hybrid Matching — Figure 3, Jia, Ding et al., CC BY 4.0.

H-DETR's Table 13 compares several setups side by side. These are COCO AP after 12, 24, and 36 epochs of training, with FPS measured on a V100:

Table: Compiled from Table 13 of H-DETR v3.
TrainingOutput used at inferenceNMS attachedAP (12 / 24 / 36 ep)FPS
One-to-many onlyOne-to-manyYes49.4 / 50.2 / 48.85.3
HybridThe one-to-many groupNo13.5 / 13.1 / 12.96.5
HybridThe one-to-many groupYes48.6 / 49.8 / 49.95.4
HybridThe one-to-one groupNo48.7 / 49.9 / 50.06.7

The second row is the one-to-many output without NMS, where AP falls to around 13 as duplicate boxes drag the whole score down. Attach NMS to the same model and it climbs back to about 49; the one-to-one group, with nothing attached, scores about the same and runs fastest. Training with one-to-many alone and then adding NMS is slightly ahead at 12 epochs, but falls behind by 36 epochs.

The other routes take different approaches but share the same shape:

  1. Add practice problems. DN-DETR feeds noised annotation boxes into the decoder as extra slots and asks the model to restore the original boxes; this practice is removed entirely at inference. DN-DETR §4.3 In 2022, DINO added negative samples and a few other improvements, and its ResNet-50 5-scale model reached 49.4 AP in just 12 epochs; with a SwinL backbone and Objects365 pre-training, it also became the first end-to-end Transformer detector to take first place on the COCO leaderboard. DINO abstract, §1
  2. Run more groups of slots. During training, Group DETR runs several groups of weight-sharing slots at once, each with its own one-to-one matching, and keeps only one group at inference. LW-DETR uses 13 groups, and the later RF-DETR carries that over. LW-DETR §3.3
  3. Put more objects in each image. At the end of 2024, DEIM left the matching alone and changed the data instead. It uses Mosaic to shrink four photos and tile them into one, or Mixup to overlay two into one, raising the average number of objects per training image from about 10 to about 25, and the positives under one-to-one matching increase with it. This augmentation is switched on for only about the first half of training; RT-DETRv2 reaches 53.6 AP in 36 epochs, where it previously took 72 epochs to reach 53.4. DEIM §3.2, Table 6, Table 12

All of this extra supervision lives only in the training phase. At inference, DETR still gives one box per object, with no NMS.

RT-DETR Times NMS Too

Once training got faster, DETR's remaining problem was slow inference. In April 2023, Baidu's RT-DETR wrote its conclusion straight into the title, DETRs Beat YOLOs on Real-time Object Detection. It was later published at CVPR 2024.

The paper first examined how YOLO speed gets measured. The authors point out that the YOLO family usually reports speed for the model alone, excluding NMS time. RT-DETR §3.1 So they built an end-to-end benchmark that attaches TensorRT's NMS plugin after YOLO and times the two together, using the thresholds from validation. On a T4 GPU with TensorRT FP16, RT-DETR-R50 reaches 53.1 AP at 108 FPS; under the same benchmark, YOLOv8-L gets 52.9 AP at 71 FPS. RT-DETR Table 2

RT-DETR didn't escape the pattern from the previous section either; its decoder also carries 200 denoising slots during training. RT-DETR Appendix Table A

On the YOLO Side, an Extra One-to-One Head

A year later, the YOLO side went a different way. In May 2024, Tsinghua University's YOLOv10 kept the original one-to-many output head and attached an extra one-to-one head. Both heads learn together during training; at inference, the one-to-many head is dropped and only the one-to-one head produces output, with no need for NMS. YOLOv10 §3.1

The left panel shows the shared backbone and two heads; the right compares how often their candidate selections agree.

A consistent matching metric improves agreement. Both heads participate in training; inference retains the one-to-one head. YOLOv10: Real-Time End-to-End Object Detection — Figure 2, Wang, Ao et al., CC BY 4.0.

For the two heads to learn well together, their criteria for picking positives have to agree. YOLOv10 has both heads choose positives with the same scoring, the one-to-many head taking the top few and the one-to-one head only the top one, so the best box in the one-to-many head's eyes is also the one-to-one head's positive.

Its ablation compares three approaches side by side (YOLOv8-S as the base, latency including NMS, T4, TensorRT FP16):

Table: Compiled from Table 3 of YOLOv10 v2.
Heads during trainingAPLatency
One-to-many only (NMS at inference)44.97.07 ms
One-to-one only43.42.44 ms
Both trained together, one-to-one at inference44.32.44 ms

Training only the one-to-one head gives great latency but costs 1.5 AP. Adding a one-to-many head as a training partner wins most of it back. Could it just be the extra rounds of computation? The authors checked that too, training a YOLOv10-M with only the one-to-one head for 550 epochs, which took about as long as the original run, and AP was still 48.9 against 51.1. YOLOv10 Table 15 More epochs can't make up the gap; it takes one-to-many supervision.

The gap doesn't fully disappear either. The authors state plainly that on small models the one-to-one head still loses to one-to-many plus NMS, by 1.0 AP on YOLOv10-N and 0.5 on YOLOv10-S; only at YOLOv10-X is there no visible gap. YOLOv10 §4.3, Appendix A.9

YOLO26 continues down this road. It was previewed at YOLO Vision 2025 on September 25, 2025, officially released on January 14, 2026, and its paper was uploaded in June. The authors argue that YOLOv10 gives the two heads the same weight from start to finish, which leaves the one-to-one head, the only one used at inference, undertrained. YOLO26 switches to progressive weights, starting the one-to-many and one-to-one losses at 0.8 to 0.2 and shifting them linearly to 0.1 to 0.9 as training goes on. YOLO26 paper §3.3.2, §4.1

The authors even tested where the weights should start. Setting the one-to-one head's weight to 0 at the start ends up about the same as a fixed half-and-half split; the best result comes from the default schedule, which starts at 0.8 to 0.2 and gradually shifts the emphasis to the one-to-one head. YOLO26 paper Table 5 DEIM also turns off its extra positives halfway through training, so both sides go dense first and sparse later.

Figure: Compiled for this article. Both lines give extra positives during training and keep only one box per object at inference.

When Measuring Speed, Does NMS Count?

The most direct benefit of dropping NMS is speed, but “how much faster” is hard to measure consistently. The same YOLOv8-N has a total latency of 6.16 ms under YOLOv10's benchmark, of which the model itself accounts for only 1.77 ms and most of the rest is NMS; YOLOv10 uses the validation thresholds, where the confidence threshold is low and many candidates remain. YOLOv10 Table 1 Under RF-DETR's benchmark, with the confidence threshold raised to 0.01, the same model's total latency is 2.1 ms. RF-DETR Table 2

When Roboflow released RF-DETR in March 2025, its launch post said that most YOLO evaluations use NMS to compute accuracy yet leave NMS out when reporting speed. Roboflow, 2025-03-20 The later RF-DETR paper listed two more problems, GPUs throttling during back-to-back inference and latency measured with an FP16 model while accuracy is measured with an FP32 one; the authors advocate measuring both with the same model artifact. RF-DETR §4 RF-DETR uses DINOv2 as its backbone, and its 2XL version runs at 17.2 ms on a T4 with 60.1 AP; the authors say it is the first real-time detector to exceed 60 AP on COCO. The paper was later published at ICLR 2026.

When reading these comparisons, also keep in mind where each party stands. Roboflow and Ultralytics are direct competitors. When the YOLO26 paper was discussed on Hacker News, a user who identified as a Roboflow co-founder objected that the paper claimed the best accuracy–latency trade-off in its headline figure while leaving recent DETRs out of the comparison. Hacker News, 2026-06-23 The headline Figure 1 compares YOLO generations, PP-YOLOE+, DAMO-YOLO, and RTMDet; the DETR family appears only in Table S8 of the appendix, and RF-DETR isn't there either. In the other direction, Roboflow's January 2026 post comparing RF-DETR and YOLO26 cites the YOLO26 numbers from the official table, 53.1 AP and 4.7 ms; according to a footnote in Ultralytics' docs, that accuracy comes from one-to-many plus NMS, while the latency was measured on the one-to-one head without NMS. YOLO26 docs, Roboflow, 2026-01-15

Ultralytics' own end-to-end guide also recommends comparing accuracy and latency with the same output head, image size, precision, and hardware. Ultralytics end-to-end detection guide

Why NMS Came Back

Back to the default change from the opening. Across all five model sizes, YOLO26's one-to-one head scores 0.6 to 0.8 AP below one-to-many plus NMS. YOLO26 paper §4.4 The paper already noted at the time that if the target platform can run NMS and accuracy matters most, the one-to-many head can still be the first choice. With 8.4.142, Ultralytics went ahead and made it the default.

Deployment is part of the reason too. According to Ultralytics' guide, platforms such as NCNN, RKNN, Edge TPU, and Qualcomm QNN fall back to the one-to-many head when their operators can't support end-to-end output, and so do some INT8 TensorRT configurations. Ultralytics end-to-end detection guide

Duplicate boxes haven't fully disappeared either. The DINO authors themselves admit that the matching loss and self-attention can only do so much to suppress duplicates. DINO §3.3 User reports have kept coming all along. In June 2024, someone fine-tuned YOLOv10-M on their own data, still had near-identical duplicate boxes after one to two thousand epochs, and finally solved it by adding NMS back. THU-MIG/yolov10 #236 In July 2026, answering a similar report about YOLO26, an Ultralytics collaborator said that YOLO26n can sometimes output duplicate boxes and that NMS can be applied if needed, since “it should be fast for few boxes.” ultralytics #25304 RF-DETR's first author gave the same advice, adding that with enough data, the model should learn not to do this on its own. rf-detr #187

In crowded scenes, how many boxes a one-to-one detector can output depends on how many slots it has. On CrowdHuman, DETR with only 100 slots scores 75.9 AP, far below an FPN detector that uses NMS (85.8); switch to Deformable DETR with 1,000 slots, and it reaches 91.5. Iter-E2EDET Table 2 When there are many objects, check first whether there are enough slots.

NMS is still here; only its place has changed. It has gone from a step inference must always run to an option you can switch on or off.

Back to Your Own Detector

To wrap up these six years:

  1. Giving each object just one prediction is expensive in training. One-to-one matching removes the need for NMS at inference, at the cost of too few positives and slow convergence.
  2. Both lines add positives during training. The DETR side adds a one-to-many branch, denoising practice, or multiple slot groups; the YOLO side keeps a one-to-many head as a training partner and switches to the one-to-one head at inference.
  3. NMS becomes an option. The one-to-one head skips post-processing; one-to-many plus NMS still gains a few tenths of an AP and is easier to deploy across platforms.

If you're choosing a detector, start by pinning down three things. When comparing speed, check whether NMS is included and which thresholds are used; when photos hold many or tightly packed objects, look at the slot limit first and keep NMS if needed; when deploying to phones or edge devices, check first whether that platform can run the one-to-one output.

The bet DETR placed back then is still playing out.

So, from where an engineer on the front line stands, can NMS be deleted?

Either way, we wouldn't dare.