← back to test matrix

Test 0.3

Can the model's confidence tell us which lines to trust?

Yes, strongly. When confident, the model is right 100% of the time; in its least-confident fifth, only 27%.
What it means for SailScan

We can trust the confidence number. Auto-accept the confident lines; route only the shaky ones to a human or the big-model backup, instead of reviewing everything. It also gives customers a real confidence badge.

100%
right when confident
27%
right when least confident
1,286
lines checked
The detail
Confidence groupLinesLand on a real stripe
Q1 (lowest)25727%
Q2 (mid)25797%
Q3 (mid)257100%
Q4 (mid)257100%
Q5 (highest)258100%

Every line split into five equal groups by the model's own confidence, then checked against real human stripes.

Source: 332-photo holdout, frozen 2026-08-25. 1,286 detections, quintile buckets.

raw data (JSON) →