{
  "_meta": {
    "test": "1.4 multi-prompt SAM3 redraw",
    "jira": "SAIL-160",
    "generatedAt": "2026-09-07",
    "generatedBy": "code/API investigation, no GPU used on this machine (per task resource limits)",
    "holdoutFile": "server/inference/test-holdout-ids-canonical.json",
    "holdoutMeta": {
      "frozen": "2026-08-25",
      "count": 332,
      "note": "frozen, read-only, never trains; referenced here only for the identity of the numbers quoted from the prior run, not re-scored in this test"
    },
    "priorNegativeSource": "server/inference/runs/hybrid-sam3-v26-deep20-20260902/score-report.json, key \"prompted-hybrid\", read directly from the file (not from memory recall)",
    "filesInvestigated": [
      "server/inference/sam3_multiprompt_predict.py (uncommitted, present in working tree)",
      "server/inference/pod-sam3-multiprompt.sh (uncommitted, present in working tree)",
      "server/inference/multiprompt_score.mjs (uncommitted, present in working tree)",
      "server/inference/runs/gate1-multiprompt-smoke.log (pod-side gate smoke-test output, dated today)",
      "server/inference/sam3_verify_pass.py (shared point-prompt/crop/gate helpers reused by the multiprompt script)",
      "ultralytics wheel 8.4.142 (the exact version pod-sam3-multiprompt.sh pins), downloaded and inspected via AST/source read on this machine -- NO torch/model weights loaded, NO GPU used, NO inference run",
      "ultralytics 8.4.84 (already installed in server/inference/.venv, inspected the same way for cross-check)"
    ]
  },
  "step1_mask_prior_support": {
    "question": "Does our SAM3 sidecar/predict interface accept a mask prior and a box prompt, or only point prompts?",
    "answer": "YES -- SAM3Predictor (the class Ultralytics selects for any checkpoint whose filename contains \"sam3\", i.e. sam3.pt) genuinely accepts and consumes points + a box + a mask prior together, in both the pod-pinned ultralytics==8.4.142 and the locally installed 8.4.84.",
    "verificationMethod": "Independent source-level re-verification on this machine: downloaded the 8.4.142 wheel (`pip download ultralytics==8.4.142 --no-deps`, no dependencies, no torch weights) and read the actual class/method bodies with Python's ast module -- did not trust the pod smoke-log's own claim at face value, traced the call chain by hand instead.",
    "callChain": [
      "sam3.pt filename contains \"sam3\" -> SAM.task_map picks SAM3Predictor (not SAM3SemanticPredictor, which is a different, box/text-grounding-only subclass used only when explicitly requested -- easy to confuse with SAM3Predictor since it shares 'SAM3' in its name; ruled out because sam3.pt is loaded via the plain SAM class, not the semantic variant)",
      "SAM3Predictor.get_model() calls build_interactive_sam3(), which builds an SAM2-family interactive/promptable SAM3Model (mask-memory architecture, use_mask_input_as_output_without_sam=True) -- this is the point/box/mask-promptable variant, not the text-grounding one",
      "SAM3Predictor defines only setup_model/get_model itself (confirmed via AST dump of both 8.4.142 and 8.4.84) -- it does NOT override inference, _inference_features, or prompt_inference",
      "SAM3Predictor also does not inherit an inference() override from SAM2Predictor (SAM2Predictor likewise defines no inference() of its own) -- so calling the predictor resolves via MRO to Predictor.inference(im, bboxes=, points=, labels=, masks=, multimask_output=, ...)",
      "Predictor.inference: 'if all(i is None for i in [bboxes, points, masks]): return self.generate(...)' else 'return self.prompt_inference(im, bboxes, points, labels, masks, multimask_output)' -- confirmed by reading the method body directly; masks/points are not silently dropped into an unused **kwargs anywhere on this path",
      "Predictor.prompt_inference -> Predictor._prepare_prompts (overridden by SAM2Predictor) which: (a) converts points/labels to tensors, (b) if bboxes present, reshapes to (N,2,2) box corners and assigns them point-labels [2,3], concatenating them onto the SAME points/labels tensor already carrying the point prompts -- this is how the box prompt is threaded in, not a separate box-only path, (c) letterboxes the mask array to the destination shape and casts to a tensor -- no manual 256x256 logit conversion required, plain (H,W) uint8 works",
      "_inference_features (SAM2Predictor's override) calls self.model.sam_prompt_encoder(points=points, boxes=None, masks=masks) -> produces real sparse_embeddings/dense_embeddings that feed self.model.sam_mask_decoder(...) -- masks genuinely reach the mask decoder as a dense prompt embedding, this is not a stubbed/ignored kwarg",
      "This is NOT the SAM2-video mask_inputs path (a different, mutually-exclusive code path elsewhere in the same file that asserts point_inputs is None) -- the image-predictor path used here has no such exclusivity constraint"
    ],
    "podSmokeLogCorroboration": {
      "file": "server/inference/runs/gate1-multiprompt-smoke.log",
      "content": "ultralytics SAM3Predictor ok; GATE OK: prompt_inference accepts ['self', 'im', 'bboxes', 'points', 'labels', 'masks', 'multimask_output']",
      "note": "this pod-side smoke check only introspects the prompt_inference signature; it does NOT by itself prove the method is reachable/consumed rather than dead code. This investigation independently traced the full call chain (inference -> prompt_inference -> _prepare_prompts -> _inference_features -> sam_prompt_encoder) to confirm the signature is not vestigial for SAM3Predictor. Both checks agree."
    },
    "caveatRuledOut": "SAM3SemanticPredictor (a DIFFERENT SAM3 subclass, used for text/box grounding only) genuinely does drop points/masks -- its own inference()/_inference_features() take only bboxes/labels/text and route through forward_grounding(), never touching sam_prompt_encoder or masks at all. This investigation initially traced that class by mistake and would have reported the opposite (wrong) answer; corrected after checking which class sam3.pt actually resolves to via SAM.task_map's is_sam3 check."
  },
  "step2_richer_prompt_vs_points_only": {
    "status": "NOT RUN -- no GPU pod reachable in this session",
    "reason": "Task constraints explicitly forbid running SAM3 locally (prior local run OOM'd this Mac) and require stopping honestly if no pod is reachable rather than attempting a local run. `runpodctl pod list` was checked: all prior pods (sam3-sailmask-smoke-test4/5, sam3-prompted-smoke) show STATUS=EXITED; none is currently running. Launching a new GPU pod was not authorized as part of this investigation task, so none was started.",
    "harnessBuiltButUnexecuted": {
      "predictScript": "server/inference/sam3_multiprompt_predict.py",
      "podLauncher": "server/inference/pod-sam3-multiprompt.sh",
      "scorer": "server/inference/multiprompt_score.mjs",
      "gitStatus": "all three files are untracked/uncommitted in the working tree -- present, reviewed, not modified by this investigation",
      "designSummary": "Runs both arms (points-only vs points+box+mask) through the identical crop/coverage/area/usable gates already validated in sam3_verify_pass.py, on the full frozen-332 holdout, scored with the same score_honest.mjs greedy matcher and isGeometryViolation gate every other eval in this repo uses. The only delta between arms is what is passed into the SAM3Predictor call, so any outcome difference is attributable to the prompt, not the scoring path.",
      "readyToRunCommand": "on a reachable 48GB+ GPU pod: scp sam3_multiprompt_predict.py + pod-sam3-multiprompt.sh to /workspace, then `bash pod-sam3-multiprompt.sh` (needs CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKEN, GH_TOKEN env vars); pull results and run `node multiprompt_score.mjs` locally to score."
    },
    "priorClosedNegativeForReference": {
      "source": "server/inference/runs/hybrid-sam3-v26-deep20-20260902/score-report.json, key \"prompted-hybrid\" (read directly from the file this session, not from memory)",
      "holdout": "test-holdout-ids-canonical.json, frozen 2026-08-25, 332 photos",
      "ruler": "score_honest.mjs greedy matcher at capPct=3 (this is the SAME ruler multiprompt_score.mjs uses -- numbers below are directly comparable to any future run of that script)",
      "maeKpPx": 36.4,
      "proximityRecall": 0.396,
      "proximityPrecision": 0.841,
      "deep20MaeKpPxSameRun": 23,
      "deep20ProximityRecallSameRun": 0.911,
      "note": "This is the points-only SAM3-redraws-every-Deep20-stripe arm being re-tested by Test 1.4 -- distinct from that same run's plain fusion 'hybrid' arm (Deep20 lines, SAM3 substituted only within-cap) which scored much closer to Deep20 (maeKpPx 22.6) precisely because it discards SAM3 lines outside the cap rather than trusting SAM3 everywhere."
    }
  },
  "verdict": "Step 1 (the pre-condition for the redesigned test): CONFIRMED SAM3 (via sam3.pt / SAM3Predictor) genuinely accepts points + a box + a mask prior together and feeds all three into the real prompt encoder -- so the richer-prompt idea IS testable as designed; it is not blocked by an API limitation. Step 2 (whether the richer prompt actually flips the known points-only negative from worse-than-Deep20 to better) has NOT been run: no GPU pod was reachable/authorized in this session. The harness (predict script + pod launcher + scorer) exists, is reviewed, and is ready to run the moment a pod is available -- this is a 'needs a pod for numbers' status, not a negative result and not a blocked design."
}