← back to test matrix

Test AH-4

Does bolting a big-model rescue onto our pipeline make the final scan better?

Yes, substantially on hard photos. Recall rises from 68.0% to 93.0% (+25.0 points) after Claude rescues stripes our model missed, at a cost of 0.24 extra junk lines per photo.
What it means for SailScan

This is the full end-to-end test: run our model, find what it missed, ask Claude to draw exactly those missing stripes, filter Claude's answers through our sail rules, and measure the COMBINED accuracy — not just Claude's accuracy in isolation. On the photos where our model already struggles, a rescue step meaningfully closes the gap. Projected across the whole 332-photo holdout (where most photos have no miss to begin with), the lift is smaller (about +3.3 points) because misses are a minority of all stripes — but on exactly the scans that need help today, this works.

68.0%
model alone
93.0%
model + Claude rescue
+25.0pts
lift
The detail
StepStripes foundRecall
Our model alone193 of 28468.0%
+ Claude rescue on the misses+71 of 91 missed78.0% of misses recovered
Combined final264 of 28493.0%

Recall lift: +25.0 percentage points (36.8% relative improvement)

Cost: 0.24 junk lines added per photo by the rescue step.

This is the hard-photo subset (87 photos where the model missed at least one stripe by construction) — not a random sample of all scans. Whole-holdout projection (recall only, not a direct measurement): not available

Source: 87 hard photos (of an ~88-photo AI-recovery test set), 2026-09-06. Whole-holdout figure is a projection from gate0-ablation-scoreboard.json, not a re-measurement.

raw data (JSON) →