Matching benchmark
How often each method finds a copy of a photograph after a common edit, and how often it matches photographs that are not the same. Run 2026-10-10 on 200 public-domain photographs with 19 edits each, against 800 unrelated photographs.
The audit matcher found 99% of copies, and 97% of cropped copies, with 0 false matches in 3,200,000 unrelated pairs.
On crops, ISCC Content-Code found 1% and Bakau's hash rule alone 0%. The audit matcher is more robust on crops than ISCC Content-Code in this run, and the gain comes from its copy-detection step, not from the hash: on these crops the hash rule alone is no better than ISCC; it gains only on centre squares. This is a small, synthetic test; read the limits below.
Recall, all edits and crops
Recall by edit
| Edit | Plain pHash | ISCC Content-Code | Bakau hash rule | Bakau audit matcher | Plain pHash, loose | ISCC Content-Code, loose |
|---|---|---|---|---|---|---|
| resize_50 | 100% | 100% | 100% | 100% | 100% | 100% |
| resize_25 | 100% | 100% | 100% | 100% | 100% | 100% |
| jpeg_q90 | 100% | 100% | 100% | 100% | 100% | 100% |
| jpeg_q60 | 100% | 100% | 100% | 100% | 100% | 100% |
| jpeg_q30 | 100% | 99% | 100% | 100% | 100% | 100% |
| crop_edge_10 | 0% | 0% | 0% | 98% | 6% | 7% |
| crop_centre_10 | 3% | 4% | 2% | 100% | 80% | 81% |
| crop_edge_25 | 0% | 0% | 0% | 97% | 0% | 0% |
| crop_centre_25 | 0% | 0% | 0% | 98% | 6% | 5% |
| crop_edge_40 | 0% | 0% | 0% | 94% | 0% | 0% |
| crop_centre_40 | 0% | 0% | 0% | 97% | 0% | 0% |
| square | 2% | 2% | 100% | 100% | 11% | 11% |
| rotate_2to5 | 3% | 3% | 3% | 99% | 76% | 76% |
| colour_brightness | 88% | 89% | 88% | 100% | 100% | 100% |
| border | 0% | 0% | 0% | 98% | 4% | 4% |
| screenshot | 100% | 100% | 100% | 100% | 100% | 100% |
| combo_crop25_resize50_q60 | 0% | 0% | 0% | 98% | 5% | 5% |
| combo_rotate_colour_q60 | 3% | 3% | 3% | 99% | 76% | 75% |
| combo_square_resize25_q60 | 2% | 2% | 100% | 100% | 12% | 12% |
| All edits | 37% | 37% | 47% | 99% | 51% | 51% |
False matches
Every original and every edit was compared with every unrelated photograph: 3,200,000 pairs.
| Method | Pairs matched | 95% upper bound per pair |
|---|---|---|
| Plain pHash | 0 | 9.4e-07 |
| ISCC Content-Code | 0 | 9.4e-07 |
| Bakau hash rule | 0 | 9.4e-07 |
| Bakau audit matcher | 0 | 9.4e-07 |
| Plain pHash, loose | 44 | 1.7e-05 |
| ISCC Content-Code, loose | 46 | 1.8e-05 |
The unrelated set held no look-alikes: it is other stock photographs, not other frames of the same scenes or other photographs by the same photographer. Look-alikes are tested in a catalogue, below.
The loose rows give plain hashes 12 bits of room instead of 2, to show what that buys and costs. Matched pairs are listed in the JSON so they can be inspected.
Look-alikes: a catalogue
The full audit, run on a synthetic catalogue of 2,493 images: 1,000 public-domain originals, about half with resized, re-encoded, cropped or rotated copies, and 40 photographers with 5 different photographs each, which must stay apart. Run 2026-10-10.
- Before the containment rule: 4 false merges: groups that joined different photographs of the same scene by the same photographer, for example two photographs of one mountain range taken months apart, and two frames of Big Ben taken the same day.
- After it (a copy must lie almost entirely within the image it copies): 2 false merges remain. In each, one framing sits inside the other, so containment cannot separate them. Both carry the "same scene, check" label in the report, for a person to look at. The label also appeared on 1 correct group. 2,787 of 2,790 pairs of the same picture were joined (2,787 before).
- Hash tier first: 1,356 SSCD descriptors instead of 2,493, and 198 pairs verified instead of 270, with identical groups.
Not a held-out test: the containment rule and the "same scene, check" label were designed after studying this catalogue's false merges (their thresholds were set on a separate tuning split), so the results after the rule show the fix working on the cases it was made for. Two Commons files that are one picture (a "cropped" derivative, or a second upload of one source) count as the same picture here. A synthetic catalogue is not a real archive; data: catalogue.json.
Method
- Originals
- 200 photographs: Wikimedia Commons, Category:Images from Unsplash (CC0/PDM only).
- Unrelated set
- 800 photographs: Wikimedia Commons, Category:Images from Pixabay (CC0/PDM only).
- Derivatives
- Commons derivative files (titles marked cropped, edited or retouched; files carrying an Extracted from, Derived from or Retouched picture template) and second uploads of one stated source are left out of both sets, so one picture is never counted as two.
- Edits
- Resizes to 50% and 25%; JPEG at quality 90, 60 and 30; crops removing 10%, 25% and 40% of width and height, from the edges or around the centre; a centre square; a rotation of 2 to 5 degrees; a mild colour and brightness change; a white border; a screenshot-like downscale and recompress; and three combinations. Made and encoded in memory; no image was written.
- Methods
- Plain pHash: the full image's 64-bit pHash. ISCC Content-Code: the 64-bit image code from iscc-core. Bakau hash rule: full and centre-square pHash, both images detailed enough, private regions not refuting. Bakau audit matcher: the hash rule, or an SSCD candidate (cosine ≥ 0.3) confirmed by local features. Hashes match within 2 bits, the configured threshold.
- Hardware
- 12th Gen Intel(R) Core(TM) i7-12700H, 20 logical cores (4 threads for SSCD), Windows-11-10.0.26200-SP0, Python 3.12.10. No GPU.
- Time per 1,000 images
- digital fingerprints (ISCC, pHash, regions): 32.4 s; ISCC Content-Code alone: 7.1 s; SSCD descriptor: 97.7 s. Verification: 2,010 candidate pairs, median 0.0954 s each.
- Reproduce
python scripts/benchmark_matching.py --n 200 --held-out 800 --seed 11 --threads 4 --out out/benchmark- Data
- results.json, the committed run.
Limits
- Small and synthetic. Two Commons categories of stock-style photographs, edited by a script. Real catalogues hold scans, many frames from one shoot and heavier edits.
- Pairs, not a full audit. The recall and false-match tables score each pair on its own. The catalogue above runs the full audit, which links each image to at most five candidates and chains groups.
- One machine. Times are from the laptop above and will differ elsewhere.
- Copies, not look-alikes, mostly. Matching looks for files that are the same picture, and can occasionally join two photographs of one scene by one photographer. It never says who made an image.
- Thresholds. Every matching threshold was set before the pair benchmark above ran, and none was tuned to it. The catalogue is different: its containment rule and label were designed after its false merges were found, so its after-rule numbers are not held out.
Want this run on your own test set? See partners.