Below: the full research + test plan each matrix row links into.

Other AIs Beyond SAM3 — Researched Shortlist (2026-09-05)

From: Skippy (Nate's AI) · For: Alex (and his AI), Nate How this was produced: a 6-angle web research sweep covering lane-detection/vector-map models, thin-structure segmentation, the post-SAM3 open-vocab foundation stack, VLM pointing/grounding, pose-model upgrades, and adjacent domains plus technique tricks. The sweep surfaced 44 raw candidates; we deduped and ranked them to 8, then a separate agent adversarially fact-checked every finalist by opening the actual repo, paper, and license (several fetched checkpoints to confirm they download). A final completeness critic reviewed the whole sweep. Claims below carry the verifier's corrections rather than the researcher's optimism.

The question (Alex, 2026-09-04): "im convinced we can get better accuracy with using some kind of Ai and is there any other than SAM3 we should be looking at?"

---

The one-paragraph answer

No better "find faint painted stripes" brain sits on a shelf. The only real SAM3 successor as of Sept 2026 is SAM 3.1, a video-speed release with no accuracy gains, and 2026 chatbot/vision AIs still fail at tracing thin lines: three 2026 papers show they lose the line and jump to a nearby look-alike, which is exactly our batten problem. The closest architectural match to our problem in all of computer vision is lane detection from self-driving cars (faint painted line on a surface in, ordered point sequence out), and one proven, permissively licensed model, CLRerNet, is worth fine-tuning as a challenger finder. The two highest-value moves cost almost nothing and use no new model at all: a pretrained straight-line detector to find the boom and veto battens/rigging, and a label audit that finds our own under-traced training labels — the root cause behind the missed faint and bottom stripes.

The verified shortlist (rank = recommended order, after critic re-rank)

1. Label audit: find our under-traced training labels

• What: score every training label by how much deep20 disagrees with it (point-to-polyline distance, plus flags where the model found a stripe the label lacks); humans review the worst-scoring ones in the existing Fix External Scans queue.
• Why first: it attacks the root cause (labels that skip faint stripes) and raises the ceiling of every retrain below.
• Verified caveats: Cleanlab's ObjectLab handles boxes only, so the polyline scorer is our own custom script (Cleanlab itself has been Apache-2.0 since Jan 2026; no license issue). In-sample scoring is biased low on exactly the systematic under-tracing we suspect, since deep20 trained on these labels; treat the cheap pass as a screen, with K-fold out-of-sample retrains as the rigorous version. The best published detectors still miss up to 66% of label errors, so this prioritizes review rather than exhausting it.
• Critic's control: also score a random-100 sample blind, so "the top-100 are mostly under-traced" gets measured against the base rate instead of confirmation bias.
• Test: one script over training labels only (frozen 332 untouched); human eyeballs top-100 vs random-100.

2. CLRerNet, the lane-detection challenger finder

• What: anchor-based lane detector where each detection is an ordered polyline with a calibrated confidence (verified in the head source code, not just docs). Apache-2.0; DLA34 checkpoints download (HTTP 200 checked). https://github.com/hirotomusiker/CLRerNet
• The trick: rotate photos and labels 90° so stripes become "lanes."
• Verified caveats: row-anchor heads predict one x per row, so after rotation every stripe must be single-valued along the luff-leech axis. Mandatory pre-build step: audit the 332 labels for doubling-back curves; if that fails, the fallback is a MapTR-style ordered point-set head (MIT, weights exist, but it lives in a multi-camera repo, so we'd borrow the head design rather than fine-tune off the shelf). CULane-pretrained checkpoints carry non-commercial dataset provenance: fine for the experiment, a gate on production promotion (mitigation: run an ImageNet-initialized comparison alongside). Integration is roughly 3 scripts (converter, slot assignment, holdout adapter) and a few days.
• Test: fine-tune on 1–2k of our labels (a few GPU-hours), run the frozen 332 via holdout_suite.sh, diff faint/bottom-stripe recall vs v26 and deep20.

3. DeepLSD / ScaleLSD: boom finder plus batten/rigging veto, zero training

• What: pretrained straight-line detectors (MIT, weights verified). The boom is a long straight near-horizontal segment; battens and rigging are straight while stripes sag with camber, so detected straight segments give both a boom candidate and a principled false-positive veto.
• Verified caveats: output is straight endpoint-pair segments, which makes this a boom/veto tool and never a stripe tracer. Long booms fragment into collinear sub-segments (merge before scoring). Boom-recall scoring needs boom ground truth the 332 holdout mostly lacks (44 of 332 have boomLines), so a boom-labeling pass is the unbudgeted prerequisite, and that same labeling budget could instead add boom keypoints to the pose data. Nate and Alex decide where it goes; nobody gets both for free.
• Test: zero-shot over the 332; minutes to tens of minutes on one GPU.

4. Test-time augmentation over deep20 (multi-scale, bottom-crop, low threshold)

• What: run the existing finder several ways (2–3 scales, an upscaled bottom-third crop aimed at the bottom-stripe miss, a lowered confidence threshold) and merge stripe candidates by polyline overlap. No training; models we already own.
• Verified caveats: Ultralytics' built-in TTA flag does not run for pose models (verified in their source), so every pass is a manual predict plus inverse transform, and all fusion is custom code (WBF handles boxes only; our own v26/v13 hybrid ensemble code in app.py is the in-house precedent). Skip horizontal flip, which flips the apparent tack. Budget one day.
• Expectation, per the critic: if the label-audit premise is right and the model never learned faint stripes exist, TTA can't conjure them, so a null result here counts as evidence for the label root cause. Either way it's the cheapest test of that hypothesis; gate on the new-batten-FP count.

5. DINOv3 dense backbone: faint-stripe evidence heatmap (linear probe first)

• What: Meta's self-supervised backbone (commercial use permitted under its custom license; weights sit behind a click-through but are self-serve). Frozen features plus a linear probe classifying stripe-centerline pixels give a cheap go/no-go on whether it "sees" the faint paint our detectors miss. Success would justify building an instance-grouping head, and heatmaps could seed SAM3 prompts or veto battens by texture.
• Verified caveats: backbone only. It emits 1/16-resolution patch features, no polylines, and the grouping head is the real unbuilt cost. Run the probe at 1024px+ input, because at low resolution a failure is a resolution artifact rather than a kill. Probe photos must exclude the 332 ids.
• Test: one pod-day, ~200 labeled photos, eyeball activation on the known-miss photos.

6. clDice / Skeleton Recall loss (only if SAM3 stays in the production path)

• What: training losses that make missed centerline pixels expensive, which targets under-traced faint ends. MIT/Apache-2.0; soft-clDice is close to a drop-in.
• Why demoted, per the critic: these are mask-space losses. They can't touch the ordered-keypoint production head (deep20), only the SAM3 mask fine-tune, and our recorded negative result says stripe-line refinement error was "not truncation." A loss also learns whatever the labels say, so it must sequence after the label audit or it faithfully learns the truncation.

7. MM-Grounding-DINO: open-vocab boxes for boom and batten classes

• What: fine-tunable open reproduction of Grounding DINO (Apache-2.0 code, weights verified downloadable). Explicit "boom" and "batten pocket" classes; boxes feed SAM3 as prompts.
• Verified caveats: boxes only. A diagonal stripe's axis-aligned box covers half the sail and stacked stripes overlap heavily, with NMS-merge risk. The fine-tune needs net-new boom/batten box labels, its biggest hidden cost, and zero-shot recall on rare vocabulary like "batten pocket" will likely be poor. Half-day zero-shot screen first.

8. YOLO26-pose, the drop-in finder swap

• What: Ultralytics' current pose model. NMS-free decoding is a real mechanism for stacked nearly-parallel stripes, where IoU-NMS suppresses adjacent low-confidence instances. Same label format; the --model flag already exists in train.py; AGPL exposure unchanged from today.
• Verified caveats: the current sidecar runs default box-NMS at iou 0.7, so suppression only fires on boxes overlapping more than 70%, and the NMS-eats-stripes effect may be modest; the test measures it either way. Pod venvs must bump the ultralytics pin. Gains stay capped by label quality, hence ranked last.
• Test: retrain on the same labels with --holdout-ids, then a holdout_suite.sh diff.

What got dismissed, and why it matters

• All chatbot/VLM tracing (GPT-5.x, Gemini 3, Qwen3-VL, Molmo): three 2026 papers ("VLMs Trace Without Tracking" 2605.15672, TraversalBench 2604.10999, "VLMs have Tunnel Vision" 2507.13361) show state-of-the-art VLMs losing a thin line and switching to a nearby look-alike, and battens and rigging are precisely that. Scaling barely helps, and we found no documented production deployment of VLM curved-line geometry anywhere. Our "sloppy lines, 10x slower" experience still matches the 2026 state of the art.
• Hosted APIs (DINO-X, Grounding DINO 1.5+, Gemini trajectories, Rex-Omni): API-only or research-licensed, which means customer photos leave our infra and nothing can fine-tune or ship. Rex-Omni's ordered-point output is the one research direction worth watching (CVPR 2026), but it sits behind two research licenses.
• Top-down pose models (RTMPose/ViTPose): stripe boxes span the full image width and overlap, so each crop contains several stripes. Near-fatal geometry mismatch.
• SAM 3.1 (March 2026): a video-tracking speed release; no accuracy reason to migrate.
• DeepSolo (curved text spotting): arguably the closest architecture in vision (ordered center-line points along a Bezier, handles doubling-back curves), but its license verified as non-commercial, so it's design inspiration only.
• No prior art: nobody has published a deep-learning sail-stripe tracer; the classical tools (SailSpy 1992, AccuMeasure) are all that exist. We're building something new rather than missing a known solution.

Cross-cutting rules from the critique (adopt for every test above)

1. Holdout hygiene: carve a dev slice from training data for tuning thresholds and margins; touch the frozen 332 only for final gates; apply the 73-content-dupe exclusion (exclude-holdout-dupes.mjs) to every holdout diff.
2. Cheap data levers the sweep under-weighted: synthetic faint-stripe degradation (fade well-labeled stripes to manufacture faint training examples at zero labeling cost) and batten/rigging hard negatives. Pseudo-labeling the ~10k unlabeled uploads is real but must wait until the label audit fixes the teacher, or it bakes in exactly our blind spot.
3. The boom-labeling budget is one decision: DeepLSD scoring, MM-GDINO classes, and a pose boom-class all want the same one-time boom annotations. Decide the target once; Alex's Cape31 correction pass is the natural vehicle (docs/cape31-program-plan.md step 4).
4. Already answered internally: the critic asked whether SAM3 full-image concept prompting was ever tried as a recall source. It was: hop-eval run D (SAIL-132) is exactly that, and it found 1039 junk lines vs 777 real. The pending idea is the curvature gate (stripes sag; battens and shrouds stay straight; SAIL-134), which is also what makes DeepLSD's veto principled.

Recommended sequence (cheap to expensive, each gated on the last)

1. Label audit plus random control (script only); SAIL ticket to follow.
2. TTA day on deep20 (no training; doubles as a test of the label-root-cause hypothesis).
3. DeepLSD zero-shot boom/veto screen (needs the boom-label decision).
4. Single-valuedness audit of the 332 labels, then the CLRerNet fine-tune (the challenger-finder bet).
5. DINOv3 high-res linear probe (one pod-day).
6. YOLO26 swap when a retrain is scheduled anyway.
7. clDice/skeleton-recall only if SAM3's production role is reconfirmed; MM-GDINO fine-tune only after boom/batten boxes exist.

Every eval number quotes its id file and freeze date, per house rules. Nothing auto-promotes.

---

Added 2026-09-05: How experts combine multiple models ("layering")

Second research sweep, four angles: production cascades, ensembling, foundation-model composition, and geometric/non-model layers. It surfaced 27 patterns, and every worth-adding claim was adversarially fact-checked against its primary sources; the corrections from that verification are baked into what follows.

The six rules the field follows

1. A refiner only works on inputs like the ones it trained on. Cascade R-CNN's core finding: running the same model twice on its own output adds nothing, and running a refiner on inputs better than its training band makes them worse. This single rule explains both of our past negative results, the keypoint model redrawing its own zoomed crops and the early SAM3 refine variant. Both are the documented failure mode of the pattern done wrong.
2. Split recall and precision into different models. The medical-imaging CAD pattern: stage 1 deliberately over-proposes and a small stage-2 classifier kills the junk; the LUNA16 lung challenge institutionalized this with candidate sensitivity around 98% before filtering. Under this rule our SAM3-solo result (1039 junk vs 777 real) reads as a healthy over-sensitive stage 1 that is missing its stage 2.
3. Judge before you redo. A verifier that only scores, accepts, or rejects can never make a good output worse, while an ungated redrawer can. Gates need their own calibration check, because miscalibrated confidence routes wrongly in both directions. Google's shipped example: MediaPipe hand tracking re-runs its detector only when landmark confidence drops.
4. Downstream layers cannot add recall. The finder's proposal set is a hard ceiling. Recall for faint/bottom stripes and booms must come from an upstream over-proposer or from a geometric prior that says where to look, never from more refinement.
5. Ensemble gains come only from diversity. Re-running the same model adds zero, which is why our YOLO-on-its-own-crops failed while YOLO+SAM3 agreement works. And models trained on the same labels share blind spots, so agreement can be confidently wrong about the same faint stripe.
6. Keep the committee offline; ship one model. Netflix's $1M prize stack was never deployed, and Google distills ensembles into single models. Every proposed layer must show its marginal number on the frozen 332 with the stage bypassed, or it doesn't earn a place.

Validation: our pipeline is already the textbook stack

• Finder feeding the SAM3 crop redraw is the Grounded-SAM detector-prompts-segmenter pattern, done canonically.
• The deterministic rules layer is the domain-constraint layer, and the debuggable stand-in for a learned combiner (which the evidence says to skip).
• Disagreement-ranked QC is textbook query-by-committee, with exactly the architectural diversity the literature wants.
• The SAM3-confirmed confidence badge is the verifier pattern: judge, don't redo.
• Corrections flowing back into training amount to informal ensemble distillation.
• Record correction honored throughout: the old "SAM3 refine hurt 77% of stripes" number was retracted 2026-09-04 (biased metric, wrong checkpoint). The corrected measurement says deep20-into-SAM3 redraw-everything is a clean win. The gating experiments below are about routing and cost, with a safety margin as the bonus; they do not undo that result.

---

Added 2026-09-05: THE TEST PLAN — every candidate, sequenced

Scope: all 8 shortlist candidates above, 5 layering additions, the gaps the completeness critic found, Nate's escalation idea, and Alex's frontier-model question. Nothing here runs until Nate/Alex say go. Proposed discipline: max 2 experiments in flight at once; each posts its result to the chat when it finishes; every final number is scored on the frozen 332 (test-holdout-ids-canonical.json, frozen 2026-08-25) with the 73 content-dupes excluded; all threshold tuning happens on a dev slice carved from training data and never on the 332; no model promotes without a human call.

Phase 0: measurement only, no training, no builds (about this week, under $5)

#	Test	What it answers	Effort
0.1	Per-stage scoreboard: end-metric on the 332 with each existing layer bypassed (finder alone / +rules / +SAM3 redraw)	The ablation table every later item is judged against	½ day, CPU, existing envelopes
0.2	Label audit + blind random control (shortlist #1); the audit script is already running as of this morning	Whether under-tracing is real and how widespread: top-100 worst-scored labels vs a random-100 base rate, human-eyeballed	script done; one human hour
0.3	Confidence-to-error stratification: join per-stripe YOLO confidence with line error on the 332, plus SAM3-redraw delta per confidence band	Whether confidence predicts quality (enables gates and a customer confidence badge), and where redraw helps or hurts	½ day, inference-only join
0.4	Curve-fit residual probe (layering): robust low-order fit per stripe (Huber or leave-one-out at our ~10-point count, where classic RANSAC is statistically thin), correlate residual with error, count batten-grab flags	A free per-line quality score and a second gate signal. Today stripeMetrics' Catmull-Rom passes through every raw point, so one bad point bends the numbers we sell	1 afternoon, CPU
0.5	Candidate-pool recall check: are the faint/bottom stripes we currently miss present among SAM3-solo's 1,816 candidates?	Whether the LUNA16 filter path can add recall or only cut junk; decides 1.3	½ day, data on disk
0.6	Sail-outline registration probe (layering, sports-field pattern): outlines on the 332 via the sail-edge spike method; count missed stripes inside their predicted height band and known FPs outside the sail	Whether one global sail fit can attack all three weaknesses. Template stays loose; sails are soft, unlike soccer pitches	1 day
0.7	Model soup of deep-slot-11/15/20 (same init, same data, same sampling: the verified-correct trio, whereas v26+deep20 have different objectives): shape-assert, uniform-average, one holdout eval; recalibrate BatchNorm before calling a negative	Free accuracy at zero serving change	½ day, 1 eval
0.8	Single-valuedness audit of the 332 labels along the luff-leech axis	CLRerNet's go/no-go (3.2); failure reroutes to a MapTR-style point-set head	2 hours
0.9	Escalation stats for Nate's rule ("only run the big model when fewer than 3 stripes are found"): how often production returns fewer than 3 stripes, and what solo SAM3 finds on exactly those photos	Sizes the win of the escalation gate before building it. This is the industry's confidence-gated cascade pattern (NoScope, MediaPipe)	½ day

Phase 1: one-day builds, no retraining (gated on Phase 0)

#	Test	Gate to proceed	Effort
1.1	TTA day over deep20 (2–3 scales, zoomed bottom-third crop, lowered threshold; manual passes since Ultralytics' TTA flag doesn't run for pose; no horizontal flip; custom polyline fusion). A null result counts as evidence for the label root cause	proceed regardless; judge on recovered misses vs new batten FPs	1 day
1.2	DeepLSD zero-shot boom finder and straightness veto (collinear-merge long segments; curvature margin mandatory)	needs boom truth on the 332, so Decision D1	1 day
1.3	Junk-vs-stripe crop classifier (LUNA16 stage 2): train on SAM3-solo's 1039 junk and 777 real (photo-level split, 332 untouched), measure ROC	only if 0.5 shows the misses live in the pool; otherwise its value is FP reduction alone	1 day, minutes of GPU
1.4	Multi-prompt SAM3 redraw (the SAMRefiner idea: points, a loose box, and a mask prior mined from the polyline). Framed plainly as re-testing a closed negative with the one untested fix; first verify our SAM3 accepts a mask prior at all	run after 0.3's per-band table exists	1 day plus GPU hours
1.5	Escalation gate v1: wire Nate's fewer-than-3-stripes rule (plus low-confidence routing if 0.3 shows signal) to trigger solo SAM3 plus the 1.3 filter only on flagged photos	0.9 sizes it; 1.3 supplies the filter	1–2 days

Phase 2: data work (gated on 0.2's audit results)

• 2.1 Worst-first relabel pass: the audit ranking feeds the existing Fix External queue; Alex sets batch size. This is the root-cause fix everything else inherits.
• 2.2 Synthetic faint-stripe degradation and batten hard negatives (the critic's catch): fade well-labeled stripes to manufacture faint training examples at zero labeling cost; mine batten/rigging crops as explicit negatives.
• 2.3 Pseudo-labeling the ~10k unlabeled uploads, only after 2.1 fixes the teacher and with committee-disagreement QC, or it bakes the blind spot in harder.

Phase 3: training bets (each is one run plus dual-ruler proof; gated on Phase 2)

• 3.1 YOLO26-pose swap (bump the pod ultralytics pin; measures the NMS-free hypothesis; rides the next retrain anyway).
• 3.2 CLRerNet fine-tune (rotate-90 converter, roughly 3 scripts; run an ImageNet-initialized arm alongside the CULane checkpoint so promotion stays license-clean), or the MapTR-style ordered point-set head if 0.8 fails.
• 3.3 DINOv3 linear probe at 1024px+ (lower resolution makes a failure meaningless); build the grouping head only if the probe passes.
• 3.4 PoseFix recipe for the SAM3 layer: retune the redraw on deep20's real output distribution, including already-correct-input-to-identity pairs. This is the literature's fix for refiners that always change something.
• 3.5 clDice / skeleton-recall loss in the SAM3 fine-tune, only if the SAM3 mask lane is confirmed staying in the production path.

Phase 4: system-level

• 4.1 Temporal burst agreement on the Cape31 boom-cam bursts: a stripe present in frames N−1 and N+1 but absent in N triggers a targeted SAM3 re-prompt. Never averaging, and gated on same-trim-state. Lives inside the Cape31 program.
• 4.2 MM-Grounding-DINO fine-tune with explicit boom and batten classes; blocked on Decision D1's boxes.

Not doing, and why (one line each)

VLM line-tracing (three 2026 papers show them losing the line to a nearby look-alike, exactly battens); hosted APIs like DINO-X and Gemini trajectories (photos leave our infra and nothing can fine-tune); WBF-style averaging fusion (selection beats fusion while confidences are uncalibrated); a learned combiner (the Netflix lesson; our rules layer already plays that role, debuggably); deep ensembles in serving (artifact count, though the snapshot variant is fine offline for committee QC); top-down pose models (stripe crops overlap near-fatally); migrating to SAM 3.1 (a video-speed release).

Answers folded in

• Nate's "fewer than 3 stripes, ask the big model" idea is the industry's confidence-gated escalation pattern, with shipped precedent in MediaPipe. It's formalized as 0.9 (size it) and 1.5 (build it).
• Alex's frontier-model question ("Fable/Opus spot stripes easily; can we use that tech?"): those are vision-language models. The measured 2026 evidence says they understand the scene yet cannot trace a thin line without jumping to a look-alike, and we found no production deployment of VLM curved-line geometry anywhere. Their honest role here is judge rather than artist: count the stripes and flag junk or a badly drawn line. That slots into 0.9/1.5 as the escalation-path checker and into QC as a second committee member, it respects rule 3, and it's cheap to test.

Decisions needed before the gated items

• D1, the boom-label budget (one annotation pass, three consumers: DeepLSD scoring, MM-GDINO classes, a pose boom class). Natural vehicle: Alex's Cape31 correction pass (docs/cape31-program-plan.md §4).
• D2, concurrency: proposed max 2 experiments in flight at any time.
• D3, reporting: each result posts to the chat as it lands, with a weekly roll-up of the 0.1 scoreboard so progress reads as one table instead of a feed.