← back to test matrix

Test 0.5

Are the stripes we miss hiding in SAM3's raw candidate pool?

Yes, about half. 29 of 58 missed stripes (50.0%) are present in SAM3-solo's over-proposed pool — a filter could recover them. The other half are genuinely absent; no filter alone will find those.
What it means for SailScan

This sets the ceiling for a junk-filter approach (test 1.3): at best it can win back about half of today's misses, not all of them. That's still real accuracy on the table, worth building — but the other half needs a fundamentally better proposer, not just a smarter filter. Greenlit test 1.3 with that ceiling in mind.

50.0%
of misses recoverable
29
recoverable stripes
29
truly absent
The detail
Stripe positionMissed by our modelSitting in SAM3's pool
Top148
Middle146
Bottom3015
Total5829

SAM3-solo draws far more candidate lines than we keep (an "over-propose" pool of 2,838 lines across 329 photos). This test checks: of the stripes our production model misses entirely, how many of them are hiding somewhere in that oversized pool, just waiting on a filter to pick them out?

Source: 332-photo holdout, frozen 2026-08-25; SAM3-solo pool from runs/hopeval/D-sam3-solo.

raw data (JSON) →