Back to paper library
arXiv 2026CVPR 2026Object trackingPaper notes

OA-SORT

Front Row, Crouch Down

Original paper

Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking

Paper authors

Chunjiang Li · Jianbo Ma · Li Shen · Yanru Chen · Liangyin Chen

Publication
CVPR 2026Conference
First submitted to arXiv
arXiv:2603.06034
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{multiobjecttrackingoasort2026,
  title = {{Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking}},
  author = {Chunjiang Li and Jianbo Ma and Li Shen and Yanru Chen and Liangyin Chen},
  year = {2026},
  eprint = {2603.06034},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.06034}
}
On this page

Two people of similar build walk past, one in front of the other, and the one behind is mostly blocked. At moments like this, the box the detector gives often covers only the small patch of the person behind that's still showing, or simply slides off onto the person in front.

The Kalman filter then updates itself with these inaccurate boxes, and its predictions go off too. When it's time to match, the detection box of the person in front may have a higher IoU with the predicted box of the person behind than with its own predicted box. The Hungarian algorithm looks only at cost, and just like that the two people swap identities.

Chunjiang Li, Yanru Chen and colleagues at Sichuan University call this phenomenon cost confusion. They first work out "who is blocking whom, and by how much," then use that to correct the matching and the Kalman filter. The whole method needs no training and can be fitted directly onto existing trackers.

Defining the problem

Position-based association has three parts: the Kalman filter predicts positions, IoU measures how well detection boxes and predicted boxes agree, and the Hungarian algorithm finds the lowest-cost matching.

The authors write both the detection box and the predicted box as "true position plus error":

d=P+Δd,e=P+Δed = P + \Delta_d,\qquad e = P + \Delta_e

Under partial occlusion, a piece of the object's appearance is missing, the detection error Δd\Delta_d grows, and IoU can jump suddenly within a frame or two. As the occlusion goes on, inaccurate detections are fed into the Kalman filter again and again, and the prediction error Δe\Delta_e accumulates too. With the two errors stacked together, the cost matrix can no longer reliably reflect who is the same person as whom.

Many trackers try to make up for this with appearance features, motion direction and detection confidence. But these cues are just as unreliable under occlusion: the appearance features extracted from a blocked person are mixed with the look of the person in front, and detection confidence gets pushed down too. So the authors go back to position itself and deal with occlusion head-on.

Solving the problem

Original Figure 2. The OA-SORT pipeline, using a group-dance scene from DanceTrack as an example. In each frame the Kalman filter first produces predicted boxes (Estimations), and OAM, together with a Gaussian map, computes each predicted box's degree of occlusion (heat maps); OAO adds the degree of occlusion to the matching cost (purple arrows); after matching, BAM uses the degree of occlusion from the previous observation to adjust the Kalman filter's update (blue arrows).
Original Figure 2, Li et al., from PDF page 3, arXiv license, scholarly quotation.

OA-SORT consists of three parts. OAM estimates the degree of occlusion, OAO uses it in matching, and BAM uses it in the Kalman filter's update.

OAM: who's in front, and how much is blocked

Original Figure 3. An illustration of depth order. The bottom edge of the image is the baseline, and the vertical distance from each box's bottom edge to the baseline is written P̂d. Box 1's bottom edge is closer to the baseline than box 2's, so person 1 is closer to the camera and stands in front of person 2.
Original Figure 3, Li et al., from PDF page 4, arXiv license, scholarly quotation.

To know who's blocking whom, you first need to know who's in front. A typical surveillance camera shoots down at an angle from above, so for people standing on the ground, the closer their feet are to the bottom of the image, the closer they are to the camera. Just comparing the bottom edges of two boxes is enough to put them in front-to-back order. To avoid misjudgments caused by jittery boxes, the bottom edges have to differ by at least 5 pixels to count.

Once the order is known, the fraction that is blocked is "the area of intersection with all the boxes in front of it" divided by "its own area":

Oci=A(Oi)A(Di)Oc_i = \frac{A(\mathcal{O}_i)}{A(\mathcal{D}_i)}

A box isn't all person, though. When a dancer spreads their arms or takes a big step, the edges of the box are mostly background, and being blocked at the edges doesn't matter much. The authors add a Gaussian map that weights each pixel by its distance from the box center: 1 at the center, decreasing outward. The blocked area becomes a weighted sum instead.

How fast the weight decays is tuned per dataset. Under the DanceTrack setting, the weight at the left and right edges of a box is only e−2.25≈0.105e^{-2.25} \approx 0.105. The same strip blocked at the edge counts for only about a tenth as much as one blocked in the middle.

OAO: those who are blocked give way in matching

OAO adds the degree of occlusion to the matching score:

S=τ(1−Oc^X)+(1−τ) IoU(D,X)S = \tau\left(1 - \hat{Oc}^{X}\right) + (1 - \tau)\, IoU(D, X)

Here the degree of occlusion is computed on the Kalman filter's predicted box XX, not on the detection box. The bottom edge of a blocked detection box often jumps around, which actually makes the front-to-back order harder to judge; predicted boxes are smoother.

The more heavily a track is blocked, the more its scores against every detection box get discounted. When two tracks compete for the same detection box, the one standing in front, unblocked, has the edge.

Take an example. Dancer j stands in front, and dancer i behind is 60% blocked. In this frame j's detection box is djd_j; its IoU with i's predicted box is 0.70, and with j's own predicted box only 0.65, because j's prediction has already been pulled off course by errors from the previous few frames. τ\tau is set to DanceTrack's 0.15:

Predicted boxOcclusionIoUIoU onlyWith occlusion
i, behind0.60.700.700.15 × 0.4 + 0.85 × 0.70 = 0.655
j, in front00.650.650.15 × 1 + 0.85 × 0.65 ≈ 0.703

Worked example for this note.

Looking only at IoU, djd_j would go to i behind, and the two would swap identities. With the degree of occlusion added, djd_j goes back to j.

OAO is used only in the first stage, the matching of high-score detection boxes.

BAM: listen less to unreliable observations

A blocked person usually gets a low detection score and is only matched in the second stage. These low-score boxes aren't very accurate, and if the Kalman filter takes them at face value, its velocity estimate gets pulled off course and the next frame's prediction goes wrong with it.

Before the update, BAM first pulls the observation partway back toward the prediction:

BAM=IoU(Xt∣t−1,Zt)⋅(1−Oc^Zt−1)Z^t=BAM⋅Zt+(1−BAM)⋅HXt∣t−1\begin{aligned} \mathrm{BAM} &= IoU\left(X_{t|t-1}, Z_t\right)\cdot\left(1 - \hat{Oc}^{Z_{t-1}}\right) \\ \hat{Z}_t &= \mathrm{BAM}\cdot Z_t + (1 - \mathrm{BAM})\cdot H X_{t|t-1} \end{aligned}

The further the observation is from the prediction, or the more heavily this track was blocked last time, the smaller BAM is and the more the update trusts the prediction. The authors point out in the appendix that this amounts to multiplying the Kalman gain by BAM, with no need to separately estimate the noise covariance under occlusion.

For example, say the predicted xx coordinate is 100, the low-score detection box is at 120, their IoU is 0.5, and this track was 60% blocked last time. BAM is 0.5×0.4=0.20.5 \times 0.4 = 0.2, and the observation used for the update becomes 0.2×120+0.8×100=1040.2 \times 120 + 0.8 \times 100 = 104.

Discussion

What each part contributes

SettingHOTAAssAIDF1Time/frame
Hybrid-SORT59.444.960.79.57 ms
+ OAO59.945.660.9+1.96 ms
+ OAO, Gaussian map60.646.762.0+9.25 ms
+ OAO, BAM60.546.562.2+3.81 ms
All61.548.063.7+14.99 ms

Compiled from Table 4 of the paper, DanceTrack validation set; times exclude detection.

Together, the three parts add 2.1 HOTA. The Gaussian map has a sizable effect: adding it to OAO takes HOTA from 59.9 to 60.6. It's also where most of the cost lies: the per-pixel computation raises the tracking time per frame from 9.57 milliseconds to 24.56 milliseconds, still fast enough for real time.

The IoU term in BAM matters too. Replacing it with a fixed constant gives HOTA 61.26 at best, with 0.4; using IoU gives 61.49.

Fitted onto other trackers

TrackerHOTA beforeAfterIDF1 beforeAfter
ByteTrack47.149.051.955.9
OC-SORT54.656.554.657.6
SparseTrack55.557.858.360.2
PD-SORT58.260.457.560.8
Hybrid-SORT62.263.163.064.2

Compiled from Table 1 of the paper, DanceTrack test set, using OC-SORT's detections.

Across the four other trackers, HOTA rises by 2.08 on average and IDF1 by 3.05. SparseTrack and PD-SORT already use a "pseudo-depth" estimated from the bottom edges of boxes, and occlusion awareness still improves them. The baseline, Hybrid-SORT, improves the least, by only 0.9, since it already handles occlusion with motion direction.

On the validation set, the authors also try fitting it onto TrackTrack: HOTA goes from 59.3 to 59.9 and IDF1 from 61.1 to 62.0. TrackTrack doesn't match with the Hungarian algorithm and also uses boxes removed by NMS, yet OA-SORT stacks on top of it just the same.

Other datasets

On SportsMOT, in the setting where the detector is trained on the training set only, HOTA goes from 73.0 to 73.4 and IDF1 from 73.3 to 74.1, without any camera motion compensation at all. The authors' explanation is that occlusion relationships are relative positions between objects, which still hold when the camera pans.

On MOT17, HOTA goes from 63.6 to 64.2 and IDF1 from 78.4 to 79.1, even slightly above Hybrid-SORT with ReID added. MOTA, however, drops from 80.6 to 79.6.

When it breaks down

Original Figure 1. OA-SORT's HOTA gain over Hybrid-SORT on each video of the DanceTrack validation set (blue line), with occlusion severity as red bars. The top chart shows eight videos with less occlusion (severity below 5%), with an average gain of 2.4%, where video 0026 is negative; the bottom chart shows five heavily occluded videos (above 15%), with an average gain of 2.14%.
Original Figure 1, Li et al., from PDF page 1, arXiv license, scholarly quotation.

Most videos improve, but video 0026 in the top chart is negative. The authors note among the limitations that the whole method relies on the bottom edge of the box to judge front and back. When someone's lower body is blocked, or a dancer leaps with both feet off the ground, the bottom edge no longer shows where that person is standing, and the front-to-back order is judged wrong.

Also, the degree of occlusion looks only at the current frame and doesn't model how occlusion changes over time, so long occlusions can still make association unstable.

Conclusion

Occlusion is the main reason position-based association goes wrong, so OA-SORT starts by computing the degree of occlusion. It orders people front to back by the bottom edges of their boxes, measures how much is blocked with intersection areas and Gaussian weights, and then puts that number into both the matching score and the Kalman filter's update.

It needs no training and no appearance features, and fitted onto five different trackers, it improves both HOTA and IDF1. The price is an assumption that only holds when "feet are on the ground": the moment a dancer on stage leaps, that assumption stops working.