Back to paper library
arXiv 2024ACCV Workshops 2024Object trackingPaper notes

GTA

Grand Theft Auto

Original paper

GTA: Global Tracklet Association for Multi-Object Tracking in Sports

Paper authors

Jiacheng Sun · Hsiang-Wei Huang · Cheng-Yen Yang · Zhongyu Jiang · Jenq-Neng Hwang

Publication
ACCV Workshops 2024Workshop
First submitted to arXiv
arXiv:2411.08216
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{multiobjecttrackinggta2024,
  title = {{GTA: Global Tracklet Association for Multi-Object Tracking in Sports}},
  author = {Jiacheng Sun and Hsiang-Wei Huang and Cheng-Yen Yang and Zhongyu Jiang and Jenq-Neng Hwang},
  year = {2024},
  eprint = {2411.08216},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.08216}
}
On this page

The Deep-EIoU paper ends by owning up to a limitation. It has no offline track correction, so when a player leaves the frame for a while and comes back, it can't recognize them.

A little over a year later, Jiacheng Sun, Hsiang-Wei Huang, Cheng-Yen Yang, Zhongyu Jiang and Jenq-Neng Hwang, from the same lab, filled in that piece. GTA here stands for Global Tracklet Association and has nothing to do with the video game. Nor is it a new tracker. It's a post-processing tool that waits for an online tracker to run through the whole video, then goes back to split tracks apart and join them up.

It can be attached after any tracker. The authors tried SORT, ByteTrack and Deep-EIoU, and all three improved.

Defining the problem

The authors sort the most common errors in athlete tracking into two kinds.

One track, several people

The first is the mix-up error. Players move irregularly and often block one another. At some moment the online tracker matches a box to the wrong person, then carries on along the mistake, and several players end up mixed into one track.

Original Fig. 3. A track output by an online tracker that actually contains three different players. Purple boxes are the player in the yellow number 12 jersey, green boxes are several other players in yellow jerseys, and red boxes are the player in the dark number 24 jersey. The dots in the middle are the ReID features of each box in this track, which DBSCAN clusters into three groups: purple, green and red.
Original Fig. 3, Sun et al., from PDF page 3, arXiv license, scholarly quotation.

This figure is a real example. Under one ID, first comes the player in yellow number 12, then it switches to another person in a yellow jersey, and finally it ends up on an opponent in dark number 24. Plot the appearance features of every box in this track, and you can see them fall naturally into three clusters.

One person, several tracks

The second is the cut-off error. In pedestrian tracking datasets, people who walk out of the frame usually don't come back; players, though, often run out of shot and return a while later. The online tracker gives them a new ID, and the same person gets cut into several pieces.

Original Fig. 4. One player's track cut into four pieces; the vertical axis is the track ID and the horizontal axis is the frame number. The four pieces appear in frames 1 to 34, 38 to 48, 581 to 623 and 636 to 648, and each is given a different ID.
Original Fig. 4, Sun et al., from PDF page 4, arXiv license, scholarly quotation.

This player leaves at frame 48 and doesn't come back until frame 581, a gap of over 500 frames, or more than 20 seconds at 25 frames per second. Keeping lost tracks around longer can handle brief exits, but an online tracker can hardly wait out a gap this long.

There have already been quite a few "global linking" methods for cut-off errors. TransLink uses a network to learn whether two pieces of track belong to the same person; StrongSORT's AFLink looks only at time and position; other methods link by feature clustering, motion features or direction of movement. GTA works at the level of each box's appearance feature and handles both kinds of error at once, splitting first and then joining.

Solving the problem

Before anything else, an OSNet ReID model trained on SportsMOT extracts an appearance feature for every box in the online tracking results.

The splitter: how many people are in one track?

The splitter handles mix-up errors. For each track, it takes the features of all its boxes and clusters them with DBSCAN.

DBSCAN is a density-based clustering algorithm. If a point has enough neighbors within radius ϵ\epsilon, that region forms a cluster; points in sparse areas that can't make up a cluster are labeled outliers. The distance here is the cosine distance between features.

The authors make two changes to it:

  1. Outliers need a home too. Standard DBSCAN discards outliers, but here every box is a real detection and can't be thrown away. So once clustering is done, each outlier is assigned to the nearest cluster.
  2. The number of clusters is capped. If more than kk clusters come out, the two closest are merged again and again until kk remain, so a track doesn't get split too finely.

Of the three hyperparameters, the minimum number of samples to form a cluster is s=5s = 5, the neighbor radius is ϵ=0.6\epsilon = 0.6, and the maximum number of clusters is k=3k = 3. However many clusters come out, the track is split into that many pieces, each given a new ID. By default, the official code only splits tracks at least 100 frames long; shorter tracks don't have enough features and are kept as they are.

The connector: which tracks are the same person?

After splitting, the connector handles cut-off errors, joining up the tracks that belong to the same person.

Original Fig. 6. The tracklet connector's pipeline. On the left are N tracks. A ReID model first extracts each track's features and an N × N distance matrix is computed; the pair with the smallest distance is found, checked against the spatial constraint and merged, the features are updated, and the number of tracks becomes N − 1. The process repeats, outputting the merged tracks on the right.
Original Fig. 6, Sun et al., from PDF page 6, arXiv license, scholarly quotation.

Step one: compute distances. For any two tracks TiT_i and TjT_j, first look at the frames they appear in, Πi\Pi_i and Πj\Pi_j. If these overlap (Πi∩Πj≠∅\Pi_i \cap \Pi_j \ne \emptyset), the two tracks appear in the same frame at the same time and can't be the same person, so the distance is set straight to the maximum value of 1.

If there's no overlap, the distance is the average cosine distance over every pair of boxes from the two tracks. Writing the cosine distance between two features as

d(a,b)=1−a⋅b∥a∥ ∥b∥d(a, b) = 1 - \frac{a \cdot b}{\lVert a \rVert\,\lVert b \rVert}

then

Di,j=1NiNj∑m∈Πi∑n∈Πjd(Fmi,Fnj)D_{i,j} = \frac{1}{N_i N_j} \sum_{m \in \Pi_i} \sum_{n \in \Pi_j} d\left(F^i_m, F^j_n\right)

NiN_i is the length of track TiT_i, and FmiF^i_m is its feature in frame mm. If one track has 300 boxes and another has 200, that's 60,000 distances to compute and then average.

Step two: add a spatial constraint. The authors assume the camera position is fixed and players' movement is bounded by the edges of the field, so a player won't leave from one side of the field and come back in from the other. So where one piece of track exits and where the next piece enters can't be too far apart:

θhor=β Δmax,hor,θver=β Δmax,ver\theta_{\text{hor}} = \beta\,\Delta_{\text{max,hor}}, \qquad \theta_{\text{ver}} = \beta\,\Delta_{\text{max,ver}}

Δmax\Delta_{\text{max}} is the maximum horizontal and vertical span of all box centers in the whole video, and β\beta is between 0 and 1. If either the horizontal or the vertical distance between one track's exit point and the other's entry point exceeds its threshold, the distance is set to 1.

Step three: hierarchical merging. Each time, pick the pair of tracks with the smallest distance; as long as that distance is below the threshold α=0.4\alpha = 0.4, merge them into one, then recompute the distances between this new track and the others. The official code only checks the spatial constraint right before merging, and a pair that fails it is dropped from consideration. This repeats until no pair has a distance below α\alpha.

Discussion

Same detections, better association

On SportsMOT, the detections reuse Deep-EIoU's YOLOX; on SoccerNet, the 2023 version of the test set is used, with ground-truth boxes taken directly as detections.

TrackerHOTAAssAIDF1DetAID switches
SORT56.342.758.874.35,180
+ GTA66.559.677.474.33,547
ByteTrack63.551.870.877.83,147
+ GTA69.762.683.277.72,107
Deep-EIoU77.267.679.888.22,909
+ GTA81.074.586.588.22,737

Compiled from Table 1 of the paper, SportsMOT test set.

  1. DetA barely changes. GTA neither adds nor removes any box; it only changes box IDs, so the detection score doesn't move, and all the gain comes from association.
  2. The weaker the tracker, the bigger the gain. SORT's HOTA rises 10.2 points, ByteTrack's 6.2 and Deep-EIoU's 3.8. SORT with GTA (66.5) already beats ByteTrack without it (63.5).
  3. The IDF1 gains are especially large. SORT gains 18.6 points, ByteTrack 12.4 and Deep-EIoU 6.7. IDF1 matches identities over the whole video, and joining broken tracks back together is exactly what it cares about most.
  4. Deep-EIoU with GTA reaches a HOTA of 81.0, the best result on SportsMOT at the time the authors reported it.

SoccerNet shows the same trend. The HOTA of SORT, ByteTrack and Deep-EIoU rises by 6.8, 4.7 and 3.7 points respectively, with Deep-EIoU going from 79.4 to 83.1.

The 79.4 here differs from the 85.4 the Deep-EIoU paper reports for itself, because GTA uses the 2023 version of the SoccerNet test set and the Deep-EIoU paper used the 2022 version. The two papers' SoccerNet numbers can't be compared directly.

Splitting or joining: which matters more?

TrackerOriginalJoin onlySplit + join
SORT56.365.466.5
ByteTrack63.569.569.7
Deep-EIoU77.280.581.0

Compiled from Table 3 of the paper, SportsMOT test set, HOTA.

Most of the gain comes from the connector. Taking Deep-EIoU as an example, of the 3.8 points in total, the connector contributes 3.3 and the splitter adds another 0.5. In other words, in sports scenes, cut-off errors are far more common than mix-up errors, or at least easier to fix.

Still, the splitter is far from dispensable. On SoccerNet, ByteTrack's IDF1 with the connector alone is 78.2, and rises to 82.7 once the splitter is added. Only when the mixed-up tracks have been split apart cleanly does the connector avoid joining the wrong people along with them.

The cost left unwritten

GTA is an offline method that needs the tracking results for the whole video before it can start, so it suits post-match analysis, not live broadcasts. The paper also doesn't report running time. Judging from how the distance is defined, every pair of tracks requires distances between all their boxes, so the longer the video and the more tracks there are, the faster the computation grows. You'll have to assess this yourself.

Also, how well the whole method works depends on the quality of the ReID features. The OSNet used here was trained on SportsMOT; for other sports or other broadcast styles, it may need retraining.

Conclusion

GTA sorts athlete-tracking errors into two kinds, one track with several people mixed in and one person cut into several tracks. It splits the former apart with DBSCAN and joins the latter back up with appearance distance plus temporal and spatial constraints. Changing no boxes, only IDs, it gives three different trackers steady gains on both SportsMOT and SoccerNet, with the connector contributing the most.

Others later picked up this combination too. In early 2026, another team used Deep-EIoU for online association and GTA for track-level correction to take first place in the SoccerTrack Challenge 2025, which is filmed with a fixed fisheye camera.

During the match, an online tracker can only do its best to keep up; once the match is over, going back to split the tracks apart and join them up properly is a solution too.