CountEx Fine-Grained Counting via Exemplars and Exclusion

CoCount: every scene contains its own distractor

Class-agnostic counters do well on benchmarks where the only salient repeated object is the target. Real requests are narrower: count the black beans, not every bean; the black coffee candies, not every sweet in the pile. CoCount is built to make that distinction unavoidable: every image contains two confusable classes, each annotated with its own independent set of dots.

Because both classes are annotated, one frame yields two counting queries: count A while excluding B, then count B while excluding A. The released splits store one record per query, which is why CoCount-train has 14,834 rows over 7,417 distinct frames. Across all three splits: 7,417 train, 1,335 val and 1,334 test frames, for 10,086 in total.

Why the two-class design matters

Because both classes in a scene are annotated, the same image yields two counting problems with the same pixels and different answers. A model that ignores the exemplars and counts "everything small and repeated" is penalised on both. The pairing also gives every positive a matched hard negative for free, with no mining and no synthetic composition.

Scale by supercategory

Counts are dense, not toy

Loading it

from datasets import load_dataset

ds = load_dataset("BBVisual/CoCount-train", split="train")
ex = ds[0]
print(ex["pos_caption"], ex["pos_count"], len(ex["pos_points"]))
print(ex["neg_caption"], ex["neg_count"], len(ex["neg_points"]))

Each record carries the image, both class names, both counts, both dot sets in absolute pixel coordinates, and exemplar boxes for each class.

CountEx

CountEx lets a user express both inclusion and exclusion intent (what to count and what to ignore) through natural-language descriptions and optional visual exemplars. At its core is a Discriminative Query Refinement module, which reasons jointly over the two cues: it first identifies visual features the target and the distractor share, then isolates the patterns specific to the exclusion, then applies selective suppression to refine the counting query.

Existing prompt-based counters let you say what to count but give you no way to say what to leave out, so cluttered scenes with confusable categories produce ambiguity and overcounting. Full abstract in the paper.

CountEx on CoCount

Two settings. KC (known-category) trains on all five supercategories and tests on seen categories. NC (novel-category) holds one supercategory out entirely, measuring zero-shot transfer to unseen objects.

Generalization to other benchmarks

The same model transfers to counting benchmarks it was not trained on, including PairTally.

What each part contributes

The exclusion prompt is what moves the needle: adding T^neg to a positive-only prompt cuts NC MAE from 32.22 to 26.67, a far larger gain than adding positive exemplars alone (31.57). The paper also reports that a mismatched negative (asking to exclude "red" in a scene of black and brown candies) still helps, reaching 28.43 - worse than a relevant negative, but better than none at all.

Corrected test annotations

We found an annotation issue in the test-set evaluation labels and corrected it. The issue is confined to evaluation labels and does not touch the training annotations CountEx learns from, since training uses dot annotations. The released dataset now ships the corrected labels; the paper was submitted beforehand and reports the originals. Both are below; differences are small and the conclusions are unchanged.

For Game, 30 frames were updated but the corrected counts land very close to the originals, so MAE and RMSE are unchanged after rounding.

Prior work: PairTally

This work follows directly from PairTally (DICTA 2025), by the same authors, which asked whether current models can count what we mean, not what they see. PairTally established the problem as a benchmark: 681 controlled images, each pairing two confusable categories over the same five supercategories used here, with 50 inter-category and 47 intra-category pairs. Evaluating ten models across three counting paradigms, it found that none reliably honours a fine-grained intent.

PairTally is diagnostic and evaluation-only, and at 681 images too small to train on. CountEx takes the next two steps: CoCount scales the same paired design to 10,086 annotated frames over 97 category pairs, enough to train on, and CountEx supplies a method for the exclusion intent PairTally showed was missing. CountEx is also evaluated on PairTally, where it outperforms pre-trained specialist counters, general vision-language models, and detectors trained on CoCount (paper, Table 4).

Benchmark and code: PairTally_Benchmark

Code and checkpoints

git clone https://github.com/bbvisual/CountEx.git
cd CountEx
bash src/eval_env_setup.sh          # conda env + deps (bf16 → Ampere or newer)

export HF_TOKEN=your_token_here
cd src
bash scripts/eval/kc.sh             # known-category
bash scripts/eval/nc_food.sh        # novel-category, Food held out

Checkpoints download from the Hub automatically: BBVisual/CountEX-KC for the KC setting and BBVisual/CountEX-NC-{Food,Home,Desk,Misc,Game} for each novel-category fold. Full instructions, training recipes and the split-code mapping are in the repository README.

BibTeX

@inproceedings{countex2026,
  title     = {CountEx: Fine-Grained Counting via Exemplars and Exclusion},
  author    = {Huang, Yifeng and Nguyen, Gia Khanh and Hoai, Minh},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

If you use CoCount or build on the fine-grained counting setting, please also cite PairTally, which introduced it:

@inproceedings{nguyen2025pairtally,
  title     = {Can Current AI Models Count What We Mean, Not What They See?
               A Benchmark and Systematic Evaluation},
  author    = {Nguyen, Gia Khanh and Huang, Yifeng and Hoai, Minh},
  booktitle = {Digital Image Computing: Techniques and Applications (DICTA)},
  year      = {2025}
}