CHIRLA Leaderboard

Results on the CHIRLA benchmark for long-term person re-identification and tracking. Baselines come from the CHIRLA paper; new results are added as researchers submit them.

Loading results…

Person Re-identification

Rank-1, Rank-5, Rank-10 (CMC) and mAP for the four scenarios, averaged over subsets. Click any column to sort.

Bold: best in column, underlined: second best, among the rows shown. Tags list the training recipe, pretraining data, input size and source of the weights; rows without a community tag are baselines from the paper. Mean columns average Rank-1 and mAP over the four scenarios; the table is ranked by mean mAP by default.

Person Tracking

CLEAR MOT metrics on the full test set (8 sequences, 56 videos) and on the two occlusion scenarios. Arrows show whether higher or lower is better.

Bold: best in column, underlined: second best, among the rows shown. Paper baselines use YOLO11n detections and each tracker’s default parameters.

Benchmark Protocol

Results are only comparable when everyone follows the same rules. Full details are in the paper and the benchmark README.

Re-identification

  • Four scenarios: reappearance, multi-camera, multi-camera long-term and long-term.
  • Each training subset is the gallery and the matching test subset is the query.
  • Subsets train_0 and test_0 are reserved for tuning and excluded from evaluation.
  • Report the average over all remaining subsets of each scenario.
  • Closed-set: distractors (negative IDs) are removed from the test sets. Open-set: distractors are kept.
  • Report the input size, pretraining data and whether CHIRLA data was used for training.

Tracking

  • Tune on the two training sequences only; evaluate on the eight test sequences (56 videos).
  • Report MOTA, IDF1, MT, ML, FN, FP, IDSW and FM on the full test set.
  • For the two occlusion scenarios, match each ground-truth box to the prediction with the highest IoU (> 0.5) and report MOTA, IDF1 and IDSW.
  • Report the detector and any appearance model you use.

Submit a Result

Evaluated a method on CHIRLA? Add it to the leaderboard so others can compare against it and cite your work.

Option 1: submission form

  1. Run your method following the protocol above.
  2. Open the submission form on GitHub.
  3. Fill in the method, setup, results, and a link to your paper or code, plus its BibTeX.

We review each submission and add it, usually within a week.

Option 2: pull request

  1. Add your entry to docs/leaderboard/results.json, and your paper under papers.
  2. Run python tools/validate_results.py to check the format.
  3. Open a pull request; the same check runs automatically.

What a Re-ID entry looks like

{
  "id": "my-method-r50",
  "method": "MyMethod (ResNet50)",
  "recipe": null, "pretrain": "MSMT17", "input": "256×128", "weights": "own training",
  "cite": ["myname2027method"], "reported_in": "myname2027method",
  "kind": "community", "verified": false, "added": "2027-02-01",
  "code": "https://github.com/…",
  "results": {
    "long_term":              { "r1": 0.0, "r5": 0.0, "r10": 0.0, "map": 0.0 },
    "multi_camera_long_term": { "r1": 0.0, "r5": 0.0, "r10": 0.0, "map": 0.0 },
    "multi_camera":           { "r1": 0.0, "r5": 0.0, "r10": 0.0, "map": 0.0 },
    "reappearance":           { "r1": 0.0, "r5": 0.0, "r10": 0.0, "map": 0.0 }
  }
}

Entries marked verified have been reproduced by the CHIRLA team or come with public code and weights. If you use the leaderboard, please also cite the CHIRLA paper.

Cite