← back to test matrix

Test 1.5

Does auto-escalating broken scans to a backup model actually fix them?

Mostly yes. 14 of 24 broken scans (58%) are genuinely rescued with real stripes. 6 more hit the line-count target but with a junk line mixed in — a false rescue.
What it means for SailScan

This turns test 0.9's finding into a real fix: the sails that come out of our pipeline broken (fewer than 3 stripes) can be automatically re-run through a backup detector and mostly recovered. It cuts the broken-scan rate from about 7.3% down to roughly 3% if we score it honestly (real stripes, not just a count). The catch: don't gate success on stripe COUNT alone, since 6 of the 20 count-based rescues were actually still wrong — gate on the rules-verified truth.

14/24
genuinely rescued
7.3% → ~3%
broken-scan rate
16
junk lines added
The detail
MeasureValue
Flagged photos (fewer than 3 stripes after rules)24
"Rescued" by raw count (≥3 lines after escalation)20 of 24
Genuinely rescued (≥3 lines that are actually real)14 of 24
Real stripes added across all 24 photos24
Junk stripes added across all 24 photos16

Test 0.9 found that 24 of 332 photos (7.3%) end up with fewer than 3 stripes after our rules run. This test wires up the fix: automatically send exactly those 24 photos to solo SAM3, filtered by our sail rules, and checks whether that actually rescues them — not just hits a stripe-count target, but recovers REAL stripes.

Source: the 24 frozen-332 photos flagged by test 0.9. Offline measurement harness, no production wiring yet.

raw data (JSON) →