# First command-risk tag — acceptance packet

Engineering checks have executed on a development integration; full acceptance remains incomplete. PS is the current implementation code. UC is a proposed successor requiring RFC-0004 acceptance. This packet applies to the bounded command finding regardless of the eventual code.

## Ground truth

Positive means the response recommends executing a supported network-to-shell command or recognized encoded-payload execution. Mere mention is not a positive. Ambiguous mixed context is uncertain. A legitimate installer can still contain the supported hazardous pattern; ground truth concerns the instruction and execution pattern, not whether the publisher is a scammer.

Fixture categories: direct shell pipelines; sudo pipelines; multiline instructions; encoded Python/JavaScript payloads; executable fenced blocks; explicit warnings; explanatory quotations; historical comments; unrelated negation; mixed safe/unsafe instructions; benign downloading; non-executing decoding; Unicode; redaction before matched spans; incomplete streams.

Independent reviewers adjudicate held-out labels without seeing detector predictions. Record disagreements and rationale. Regression examples are separate from frozen held-out examples. Do not tune on the holdout; version a new evaluation corpus if it becomes development material.

## Observable acceptance checks

| ID | Scenario | Required result | Evidence / owner |
|---|---|---|---|
| A01 | Direct supported execution instruction | Finding names the pattern and exact redacted-text span | Fixture runner / Engineering lead |
| A02 | Warning, historical quotation, or explanation | No recommendation finding; ambiguity is explicit | Negative and ambiguous fixtures / Claim reviewer |
| A03 | Fenced executable recommendation; unrelated nearby negation | Valid positive is not blanket-suppressed | Boundary fixtures / Engineering lead |
| A04 | Redaction and non-ASCII characters before the finding | Subject hash and offsets match published vectors | Conformance runner / Engineering lead |
| A05 | Two responses evaluate out of order | Each result mounts only on its original unchanged revision | Forced-delay browser scenario / Engineering lead |
| A06 | Response edited, regenerated, removed, or navigated away | Stale result discarded; changed revision reevaluated | Browser scenarios / Engineering lead |
| A07 | Duplicate capture or repeated mutations | Exactly one current badge per response revision | Browser scenario / Engineering lead |
| A08 | Missing/wrong token; backend/redaction failure; malformed upstream JSON | Explicit unavailable/error; no successful check or privacy bypass | Integration scenarios / Engineering lead |
| A09 | Unsupported page/modality | Explicit unsupported; no detector invocation under false support claims | Integration scenario / Engineering lead |
| A10 | No supported pattern | No-finding wording states this is not a safety clearance | UI fixture / Claim reviewer |
| A11 | Keyboard operation | Reach/open/read/close evidence; Escape restores trigger focus | Manual script / reviewer |
| A12 | Screen reader, 200% zoom, 320px reflow, forced colors, reduced motion | Evidence remains understandable and operable; polite announcements | Automated plus human evidence / reviewer |
| A13 | Invalid tag or unknown signal | Safe text rendering, no script injection, clear unknown evidence | UI adversarial fixture / Engineering lead |
| A14 | Local pipeline and logging | Detected redactions precede evaluator; synthetic markers absent from application logs/storage | Integration and isolation checks / Engineering lead |
| A15 | Independent install/removal | Outside pilot user completes both using instructions | Consented pilot / Product owner |
| A16 | Challenge or correction | User can report tag ID/method without mandatory content disclosure; corrections reference original record | Workflow demonstration / Claim reviewer |
| A17 | Release capability selection | Only accepted enabled tags can emit; changing eval YAML alone cannot imply runtime enforcement | Gateway/evaluator test / Engineering lead |

## Statistical release gate proposal

Agree before holdout execution: two-sided 95% Wilson intervals for precision and recall; candidate lower bounds 0.93 and 0.75 respectively. Precision denominator is TP+FP; recall denominator TP+FN. Zero denominators are insufficient evidence, never a pass. Report counts, intervals, category performance, abstentions and uncertain coverage separately. Category coverage requirements and sample design must be accepted before the holdout is assembled. No MT gate blocks this tag. No score is presented as calibrated probability without a separate calibration study. A deterministic detector has no mandatory minimum abstention quota.

If a gate fails, improve the detector, narrow supported categories, or defer release. Do not lower the accepted gate retrospectively. The 100ms latency target remains provisional until hardware, input sizes, startup state and pipeline boundaries are measured.

## Result contract

Six candidate states: PENDING, FINDING, NO_FINDING, UNCERTAIN, UNSUPPORTED, UNAVAILABLE. State is distinct from tag code. Unavailable means a completed valid evaluation could not be obtained; a partial internal check may have occurred. Do not claim no check occurred in every failure case.

## Remaining product decisions

RFC-0004 rename/registration, first browser/site, statistical and category gate acceptance, signing policy, and independent adjudication ownership. These are explicit decision gates, not missing engineering specifications. No endpoint addition is authorized here.

## Execution status — 2026-10-08

These are development checks, not a declaration that every acceptance scenario passes.

| Checks | Executed evidence | Still missing |
|---|---|---|
| A01–A03 | Four development datasets and narrow context/evaluation-layer checks pass; disputed historical shell labels explicitly superseded | Independent labels, category coverage, general quoted/ambiguous-context validation |
| A04 | Published developer-generated Unicode/redaction vector passes gateway/evaluator integration | Independent conformance review |
| A05–A07 | Synthetic browser checks verify out-of-order results, text revisions, identity and navigation races, lost-role cleanup, stale-result rejection and one mount | Full live regeneration/navigation behavior |
| A08 | HTTP authentication, outage, timeout, malformed output, offsets and body limits tested | Outside-user recovery judgment |
| A09 | Gateway rejects non-text; absent anchors and missing identity produce visible unsupported status; one live assistant body verified | Broader live-site and unsupported-page coverage |
| A10 | Synthetic no-finding wording verified | Independent comprehension check |
| A11–A12 | Real keyboard evidence opening; synthetic Escape focus, axe, 320px reflow, forced colors and reduced motion pass | Human screen-reader use and zoom assessment |
| A13 | Unknown evidence ID rendered as text; injection rejected on fixture | Independent adversarial review of the complete result contract |
| A14 | Real redaction ordering, offline email recognition, bounded egress probes, source hashes, restricted containers and recent synthetic-log markers checked | Broader application/OS security review and redaction coverage evidence |
| A15 | Installation instructions and real unpacked extension exercised by developer | Outside participant installation/removal |
| A16 | Evidence panel exposes tag ID, subject and evaluator hashes, redaction categories, method and privacy-preserving challenge guidance | Accepted challenge intake, signing and correction/supersession policy and implementation |
| A17 | Runtime exclusion tests pass; config does not activate proposed codes | Product acceptance of any successor code before conformance release |

Evidence: `runs/2026-10-08-live-gap-implementation.json` links current browser,
container and real-extension checks. Sixty Python checks pass. The installed Chrome
extension produced a finding on one live Homebrew answer; its evidence panel shows
the exact rebuilt evaluator hash and redaction metadata. This verifies one response,
not broad vendor compatibility or independent accuracy. Statistical evaluation
still fails correctly. The earlier transient 504 remains historical evidence with
its cause unconfirmed; the latest three synthetic serial checks completed at
750.7, 16.9 and 12.5 ms. No latency SLA is accepted.