Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking
- Publication
- CVPR 2026Conference
- First submitted to arXiv
- arXiv:2603.06034
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{multiobjecttrackingoasort2026,
title = {{Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking}},
author = {Chunjiang Li and Jianbo Ma and Li Shen and Yanru Chen and Liangyin Chen},
year = {2026},
eprint = {2603.06034},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2603.06034}
}
Two people of similar build walk past, one in front of the other, and the one behind is mostly blocked. At moments like this, the box the detector gives often covers only the small patch of the person behind that's still showing, or simply slides off onto the person in front.
The Kalman filter then updates itself with these inaccurate boxes, and its predictions go off too. When it's time to match, the detection box of the person in front may have a higher IoU with the predicted box of the person behind than with its own predicted box. The Hungarian algorithm looks only at cost, and just like that the two people swap identities.
Chunjiang Li, Yanru Chen and colleagues at Sichuan University call this phenomenon cost confusion. They first work out "who is blocking whom, and by how much," then use that to correct the matching and the Kalman filter. The whole method needs no training and can be fitted directly onto existing trackers.
Defining the problem
Position-based association has three parts: the Kalman filter predicts positions, IoU measures how well detection boxes and predicted boxes agree, and the Hungarian algorithm finds the lowest-cost matching.
The authors write both the detection box and the predicted box as "true position plus error":
Under partial occlusion, a piece of the object's appearance is missing, the detection error grows, and IoU can jump suddenly within a frame or two. As the occlusion goes on, inaccurate detections are fed into the Kalman filter again and again, and the prediction error accumulates too. With the two errors stacked together, the cost matrix can no longer reliably reflect who is the same person as whom.
Many trackers try to make up for this with appearance features, motion direction and detection confidence. But these cues are just as unreliable under occlusion: the appearance features extracted from a blocked person are mixed with the look of the person in front, and detection confidence gets pushed down too. So the authors go back to position itself and deal with occlusion head-on.
Solving the problem

OA-SORT consists of three parts. OAM estimates the degree of occlusion, OAO uses it in matching, and BAM uses it in the Kalman filter's update.
OAM: who's in front, and how much is blocked

To know who's blocking whom, you first need to know who's in front. A typical surveillance camera shoots down at an angle from above, so for people standing on the ground, the closer their feet are to the bottom of the image, the closer they are to the camera. Just comparing the bottom edges of two boxes is enough to put them in front-to-back order. To avoid misjudgments caused by jittery boxes, the bottom edges have to differ by at least 5 pixels to count.
Once the order is known, the fraction that is blocked is "the area of intersection with all the boxes in front of it" divided by "its own area":
A box isn't all person, though. When a dancer spreads their arms or takes a big step, the edges of the box are mostly background, and being blocked at the edges doesn't matter much. The authors add a Gaussian map that weights each pixel by its distance from the box center: 1 at the center, decreasing outward. The blocked area becomes a weighted sum instead.
How fast the weight decays is tuned per dataset. Under the DanceTrack setting, the weight at the left and right edges of a box is only . The same strip blocked at the edge counts for only about a tenth as much as one blocked in the middle.
OAO: those who are blocked give way in matching
OAO adds the degree of occlusion to the matching score:
Here the degree of occlusion is computed on the Kalman filter's predicted box , not on the detection box. The bottom edge of a blocked detection box often jumps around, which actually makes the front-to-back order harder to judge; predicted boxes are smoother.
The more heavily a track is blocked, the more its scores against every detection box get discounted. When two tracks compete for the same detection box, the one standing in front, unblocked, has the edge.
Take an example. Dancer j stands in front, and dancer i behind is 60% blocked. In this frame j's detection box is ; its IoU with i's predicted box is 0.70, and with j's own predicted box only 0.65, because j's prediction has already been pulled off course by errors from the previous few frames. is set to DanceTrack's 0.15:
| Predicted box | Occlusion | IoU | IoU only | With occlusion |
|---|---|---|---|---|
| i, behind | 0.6 | 0.70 | 0.70 | 0.15 × 0.4 + 0.85 × 0.70 = 0.655 |
| j, in front | 0 | 0.65 | 0.65 | 0.15 × 1 + 0.85 × 0.65 ≈ 0.703 |
Worked example for this note.
Looking only at IoU, would go to i behind, and the two would swap identities. With the degree of occlusion added, goes back to j.
OAO is used only in the first stage, the matching of high-score detection boxes.
BAM: listen less to unreliable observations
A blocked person usually gets a low detection score and is only matched in the second stage. These low-score boxes aren't very accurate, and if the Kalman filter takes them at face value, its velocity estimate gets pulled off course and the next frame's prediction goes wrong with it.
Before the update, BAM first pulls the observation partway back toward the prediction:
The further the observation is from the prediction, or the more heavily this track was blocked last time, the smaller BAM is and the more the update trusts the prediction. The authors point out in the appendix that this amounts to multiplying the Kalman gain by BAM, with no need to separately estimate the noise covariance under occlusion.
For example, say the predicted coordinate is 100, the low-score detection box is at 120, their IoU is 0.5, and this track was 60% blocked last time. BAM is , and the observation used for the update becomes .
Discussion
What each part contributes
| Setting | HOTA | AssA | IDF1 | Time/frame |
|---|---|---|---|---|
| Hybrid-SORT | 59.4 | 44.9 | 60.7 | 9.57 ms |
| + OAO | 59.9 | 45.6 | 60.9 | +1.96 ms |
| + OAO, Gaussian map | 60.6 | 46.7 | 62.0 | +9.25 ms |
| + OAO, BAM | 60.5 | 46.5 | 62.2 | +3.81 ms |
| All | 61.5 | 48.0 | 63.7 | +14.99 ms |
Compiled from Table 4 of the paper, DanceTrack validation set; times exclude detection.
Together, the three parts add 2.1 HOTA. The Gaussian map has a sizable effect: adding it to OAO takes HOTA from 59.9 to 60.6. It's also where most of the cost lies: the per-pixel computation raises the tracking time per frame from 9.57 milliseconds to 24.56 milliseconds, still fast enough for real time.
The IoU term in BAM matters too. Replacing it with a fixed constant gives HOTA 61.26 at best, with 0.4; using IoU gives 61.49.
Fitted onto other trackers
| Tracker | HOTA before | After | IDF1 before | After |
|---|---|---|---|---|
| ByteTrack | 47.1 | 49.0 | 51.9 | 55.9 |
| OC-SORT | 54.6 | 56.5 | 54.6 | 57.6 |
| SparseTrack | 55.5 | 57.8 | 58.3 | 60.2 |
| PD-SORT | 58.2 | 60.4 | 57.5 | 60.8 |
| Hybrid-SORT | 62.2 | 63.1 | 63.0 | 64.2 |
Compiled from Table 1 of the paper, DanceTrack test set, using OC-SORT's detections.
Across the four other trackers, HOTA rises by 2.08 on average and IDF1 by 3.05. SparseTrack and PD-SORT already use a "pseudo-depth" estimated from the bottom edges of boxes, and occlusion awareness still improves them. The baseline, Hybrid-SORT, improves the least, by only 0.9, since it already handles occlusion with motion direction.
On the validation set, the authors also try fitting it onto TrackTrack: HOTA goes from 59.3 to 59.9 and IDF1 from 61.1 to 62.0. TrackTrack doesn't match with the Hungarian algorithm and also uses boxes removed by NMS, yet OA-SORT stacks on top of it just the same.
Other datasets
On SportsMOT, in the setting where the detector is trained on the training set only, HOTA goes from 73.0 to 73.4 and IDF1 from 73.3 to 74.1, without any camera motion compensation at all. The authors' explanation is that occlusion relationships are relative positions between objects, which still hold when the camera pans.
On MOT17, HOTA goes from 63.6 to 64.2 and IDF1 from 78.4 to 79.1, even slightly above Hybrid-SORT with ReID added. MOTA, however, drops from 80.6 to 79.6.
When it breaks down

Most videos improve, but video 0026 in the top chart is negative. The authors note among the limitations that the whole method relies on the bottom edge of the box to judge front and back. When someone's lower body is blocked, or a dancer leaps with both feet off the ground, the bottom edge no longer shows where that person is standing, and the front-to-back order is judged wrong.
Also, the degree of occlusion looks only at the current frame and doesn't model how occlusion changes over time, so long occlusions can still make association unstable.
Conclusion
Occlusion is the main reason position-based association goes wrong, so OA-SORT starts by computing the degree of occlusion. It orders people front to back by the bottom edges of their boxes, measures how much is blocked with intersection areas and Gaussian weights, and then puts that number into both the matching score and the Kalman filter's update.
It needs no training and no appearance features, and fitted onto five different trackers, it improves both HOTA and IDF1. The price is an assumption that only holds when "feet are on the ground": the moment a dancer on stage leaps, that assumption stops working.