YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information
- Publication
- ECCV 2024Conference
- First submitted to arXiv
- arXiv:2402.13616
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{objectdetectionyolov92024,
title = {{YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information}},
author = {Chien-Yao Wang and I-Hau Yeh and Hong-Yuan Mark Liao},
year = {2024},
eprint = {2402.13616},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2402.13616}
}
YOLOv8 didn’t publish a paper either?
If so, let's continue reading the YOLOv9 paper, which is also a work from Taiwan's Academia Sinica.
Problem Definition
In recent years, the YOLO series has become the de facto standard in real-time object detection. From YOLOv3 to YOLOv7, the continuously evolving architectures have achieved a good balance between accuracy and inference speed, widely applied in surveillance, traffic, AR/VR, and many other scenarios.
Most of these models adopt CSPNet or ELAN as backbone modules, assisted by PAN/FPN for feature fusion, and use YOLO heads or FCOS heads for multi-scale prediction. Transformer detectors such as RT-DETR also compete in real-time detection; speed and deployment comparisons depend on model size, hardware, and test conditions.
However, as models become increasingly complex, researchers gradually realize that the issues affecting convergence quality do not stem solely from architectural design or loss function choice, but deeper bottlenecks actually lie in the information transmission paths.
The real challenge of modern deep neural networks lies in the information bottleneck during training.
As input data passes through deep networks layer by layer, the original semantic information is repeatedly compressed and transformed, often diluted or lost by the intermediate layers. This semantic degradation leads to features obtained at the later stages being poorly correlated with the target, causing unreliable gradients that result in slow convergence, low accuracy, or even ineffective training.
Although the industry has proposed three major technical approaches to address this problem:
-
Reversible Architectures: Designs with inverse mapping functions that preserve complete input information after each layer operation. RevNet has an explicitly reversible design; multi-path modules such as Res2Net preserve and reuse information but are not necessarily exactly invertible. Putting a large reversible structure into the inference backbone raises computation and latency.
-
Masked Modeling: Strategies like those used in MAE or BERT, where the model learns key semantics by reconstructing masked regions. However, these methods are mostly designed for large Transformer architectures and tend to fail at stable semantic restoration when applied to lightweight models due to insufficient parameters.
-
Deep Supervision: Inserting auxiliary supervision signals into intermediate layers to guide the model to maintain semantic consistency. While this can improve learning stability, it is prone to accumulating erroneous information and hurts the smallest model in the paper’s ablation, while deeper variants can benefit.
Though each technique has merits, none can simultaneously satisfy both "information retention" and "lightweight efficiency" demands. The authors consider this bottleneck a major obstacle especially for parameter-constrained real-time models.
Why Does Semantic Information Get Lost?
Traditionally, model convergence difficulties were attributed to "vanishing gradients" or "gradient saturation," but these issues have been largely mitigated by BatchNorm, ReLU, and other techniques. This paper focuses on another issue: features used to calculate gradients may have lost task-relevant information.
According to the Information Bottleneck principle, after transformation by each network layer, the paper uses a data-processing inequality to describe how a fixed feature path cannot create new information about target :
where denotes the semantic information in the input data relevant to the target.
If this information is diluted during forward propagation, even with little total information loss, the model cannot learn truly useful target mappings. This phenomenon is especially pronounced in lightweight models, since their limited parameters cannot support full feature representation, making semantic signals more prone to compression and erasure.
Expanding model capacity can alleviate symptoms, but this "wider instead of deeper" approach cannot fundamentally solve semantic loss and gradient degradation.
The Possibility of Reversible Functions
Reversible functions offer a theoretical solution.
If a transformation function has an inverse satisfying:
then an ideal inverse preserves the information in . PGI does not claim that every layer of the inference branch is exactly invertible.
Putting a composite reversible path into the inference backbone adds computation and latency, which can make real-time deployment harder.
Therefore, we need a novel method that not only preserves key semantics and improves gradient reliability but also:
- Applies to lightweight architectures
- Incurs zero extra inference cost
- Can flexibly combine with task losses and structural modules
Considering these requirements, the authors propose a novel concept to address the information bottleneck:
Programmable Gradient Information (PGI).
Let's continue.
Solution
To solve the information bottleneck and gradient degradation issues, the authors propose a new training framework: Programmable Gradient Information (PGI).
As shown above, this framework draws on experiences from prior methods (such as PAN, RevCol, Deep Supervision), designing a training strategy that supplements semantic loss during training yet removes its training-only auxiliary branch at inference.
Design Concept
PGI consists of three key components:
-
Main Branch
- The original model architecture responsible for inference.
- Only this branch is kept during actual deployment, maintaining inference efficiency.
-
Auxiliary Reversible Branch
- Provides extra supervision and gradients during training to help the main branch retain task-relevant features. The paper proposes this as a way to ease the information bottleneck.
-
Multi-level Auxiliary Information
- Integrates gradient information from prediction heads at different scales between auxiliary supervision and the main branch, so each layer receives information about objects of different sizes rather than one scale alone.
In Figure (b), reversible architectures like RevCol can preserve information integrity but significantly increase inference latency. The authors observe that instead of making the main branch reversible, it is better to introduce an auxiliary reversible branch during training.
Figure (d) illustrates this design.
During training, the auxiliary reversible path uses multi-level connections to provide gradients and help the main branch learn features useful for detection. The diagram and ablations support this design, but cannot show that every gradient step is fully correct.
Multi-level Semantic Integration
Traditional deep supervision, as shown in Figure (c), assigns features at each layer to different tasks (e.g., small object detection, large object detection), which can cause semantic conflicts. For example, a layer focusing on small objects while ignoring large ones may cause the backbone features to mistakenly treat important regions as background, resulting in distorted training signals.
To ease this problem, PGI proposes a Multi-level Auxiliary Information module that inserts an integration network between auxiliary supervision’s multi-scale feature pyramid and the main branch, combining gradient information returned by different prediction heads.
The advantages of this design are:
- Features at all layers can learn about objects across scales rather than correspond to a single object scale only.
- Mitigates supervisory conflicts when prediction heads focus on different subsets of targets.
- Allows customizable fusion architectures to flexibly adjust semantic depth and scope.
PGI transforms “deep supervision” from simple multi-point losses into a multi-semantic consistency fusion training strategy.
Generalized ELAN
To fully unleash the potential of the PGI framework, YOLOv9 concurrently introduces a new backbone architecture:
- Generalized ELAN (GELAN).
As shown above, GELAN draws inspiration from CSPNet and ELAN, where:
- CSPNet provides branch-wise gradient control to enhance feature reuse;
- ELAN uses deep convolutional stacking for effective aggregation.
GELAN further generalizes ELAN’s computational unit design so its blocks can be replaced. The paper compares ordinary convolution with ResNet, DarkNet, and CSP blocks; this ablation does not test depthwise convolution or attention blocks.
Its advantages include:
- Modular design: allows choosing blocks for a target platform; mobile and embedded latency are not measured here.
- Stable gradient paths: structurally built-in signal flow mechanisms that synergize well with PGI.
- Lightweight yet efficient: can achieve higher parameter utilization efficiency even without using depthwise convolutions.
As the main computational architecture, GELAN is paired with PGI’s training-time auxiliary supervision; the ablations below examine their contributions separately.
Discussion
To verify the practicality and generalization capability of the proposed PGI training mechanism and GELAN architecture, the authors conducted comprehensive experiments on the MS COCO 2017 dataset, comparing against multiple current state-of-the-art real-time object detection models.
This study follows the YOLOv7 AF training protocol and uses the standard MS COCO 2017 dataset split. The authors’ GELAN/YOLOv9 models were trained from scratch for a total of 500 epochs, with the first 3 epochs using linear warm-up and subsequent learning rates decayed according to model scale.
To stabilize final training, Mosaic data augmentation was disabled during the last 15 epochs.
YOLOv9 is implemented in two versions:
- YOLOv9-C / YOLOv9-E: built respectively upon YOLOv7 and Dynamic YOLOv7.
- The original YOLOv7’s ELAN modules were replaced by the proposed GELAN, using RepConv and CSPNet blocks as computational units.
- Downsampling modules were simplified, and the anchor-free prediction head further optimized.
- PGI’s auxiliary supervision settings fully retained YOLOv7’s multi-branch feature learning design.
Comparison with Existing Methods
The table lets us compare parameter counts, computation, and AP at several model sizes. What do these rows actually support?
-
Small-model comparison: YOLOv9-S versus YOLO MS-S is 46.8 versus 46.2 AP, 7.1M versus 8.1M parameters, and 26.4G versus 31.2G FLOPs. The table does not report edge-device latency.
-
Medium-model comparison: YOLOv9-M versus YOLO MS is 51.4 versus 51.0 AP, 20.0M versus 22.2M parameters, and 76.3G versus 80.2G FLOPs. The AP difference here is 0.4 point.
-
Compared with the mainstream YOLOv7 AF, YOLOv9-C reduces parameters by 42% and FLOPs by 22%, while achieving the same AP (53%). This comparison also changes the network architecture, so the difference cannot be assigned to PGI alone.
-
Compared to the large YOLOv8-X, YOLOv9-E cuts parameters by 16% and FLOPs by 27%, yet raises AP by 1.7 points (53.9→55.6), fully showcasing the synergy of efficient design and semantics-driven training.
These COCO results show competitive AP for the authors’ YOLOv9 models trained from scratch without extra pretraining data. The table compares parameters, FLOPs, and accuracy; latency on other hardware and real deployments still need testing.
GELAN Architecture Analysis
First, the authors examined the substitutability of GELAN’s computational modules, comparing three alternatives:
- ResNet Block
- DarkNet Block
- CSP Block
Across the four GELAN-S computational-block rows, AP ranges from 44.3 to 45.5. CSP Block performs best: compared with Conv, AP rises from 44.8 to 45.5 (+0.7 point), while parameters fall from 6.2M to 5.9M and FLOPs from 23.5G to 22.4G.
Hence, YOLOv9 ultimately selects CSP as GELAN’s core unit, balancing performance and lightweight design.
Further, the authors tested the impact of ELAN module depth and CSP stacking depth on performance across different model scales.
The experiments show:
- For GELAN-C, increasing ELAN depth from 1 to 2 raises AP from 50.7 to 52.5.
- In the tested configurations, increasing depth also raises parameters and FLOPs, but AP gains do not follow a fixed slope. For GELAN-C, changing ELAN/CSP depth from 2/1 to 2/2 raises AP from 52.5 to 53.0 and parameters from 25.3M to 28.6M.
These rows support choosing among the tested depth/cost trade-offs; untested configurations and devices still need evaluation.
YOLOv9 versions use the following depth configurations:
- YOLOv9-S: ELAN depth 2, CSP depth 3
- YOLOv9-M / C: ELAN depth 2, CSP depth 1
PGI Training Strategy Analysis
PGI consists of two main components: the auxiliary reversible branch and multi-level semantic integration.
The authors conducted ablation studies independently and jointly, finding:
- The auxiliary reversible branch (ICN) uses DHLC linkage design to provide multi-level reversible information, stably enhancing model performance.
- Multi-level semantic integration was tested with FPN and PAN structures; PFH is the conventional deep-supervision comparator. GELAN-C’s PFH row ties the 52.5 AP baseline, while GELAN-E’s PFH row rises from 55.0 to 55.3.
- ICN and integration gains depend on the configuration: on GELAN-C, backbone ICN reaches 52.9 AP and LHG-ICN reaches 53.0. On GELAN-E, FPN alone and FPN plus ICN both reach 55.6 AP, so not every combination adds a further gain.
Additionally, YOLOv7’s Lead-Head Guided Assignment technique was applied to PGI’s auxiliary supervision path, further improving supervision quality and final learning outcomes.
Model Scale Comparison
The authors applied PGI and traditional Deep Supervision on different model scales, observing:
- Ordinary deep supervision lowers GELAN-S from 46.7 to 46.5 AP; GELAN-M instead rises from 51.1 to 51.2, so it does not lower every smaller model.
In Table 5, ordinary deep supervision (DS) changes S by -0.2 AP but M/E by +0.1/+0.3. PGI raises AP in all four rows, though S gains only 0.1 point.
These rows support two narrower conclusions about PGI:
- It provides a small positive gain on the lightweight S model where ordinary deep supervision slightly lowers AP.
- Its reported AP gain is larger on C/E models; more reliable gradients are the authors’ proposed mechanism, not a quantity measured by this table.
Overall Component Integration
Finally, the authors present an ablation from the YOLOv7 baseline to the full YOLOv9-E version.
This is a cumulative ablation: YOLOv7 starts at 36.9M/104.7G/51.2 AP; adding AF yields 43.6M/130.5G/53.0 AP; replacing it with GELAN yields 41.2M/126.4G/53.2 AP; adding DHLC yields 57.3M/189.0G/55.0 AP; and the final PGI step reaches 55.6 AP at the same 57.3M/189.0G. Relative to the initial YOLOv7 row, accuracy improves while parameters and FLOPs also rise.
Conclusion
The key innovation of YOLOv9 lies in transforming the "gradient flow" into a programmable information transmission mechanism.
Through PGI, training is no longer passive backpropagation but an active planning of:
- How semantics should flow
- Where gradients should come from
- At which layer the target should be learned
Together with the GELAN backbone, YOLOv9 simultaneously achieves three major goals: high accuracy, low parameter count, and deployment flexibility.