# FIVE-SIGNAL INTEGRITY INVENTORY

**Status:** LOAD-BEARING -- every run evaluates all five signals in fixed order. Any signal firing is a first-class finding, co-emitted to Self-Audit Resistance Monitor (E9).

**Inventory version:** 1.0

---

## Signal order (non-negotiable)

Signal 3 (pre-registration) is evaluated first because it gates the entire run. Signals 1, 2, 4, 5 are evaluated after probes run so their observations are concrete. The listed order matches the canonical numbering used across the Evaluator system for log consistency.

Evaluation sequence:

```
Phase A enter -> evaluate Signal 3
  if Signal 3 fires -> HALT, return to Audit Charter Lock
  else -> proceed to probes
Phase C, D run -> evaluate Signals 1, 2, 4, 5 after probe sweep
```

---

## Signal 1 -- Self-audit vs independent-audit delta > 3x

**Trigger:** if RAM's output is a self-audit artifact (RAM auditing its own prior work, or auditing a system RAM developed) and the Evaluator finds defects at more than 3x the rate RAM self-reported on the same artifact class.

**Empirical precedent:** NY State Comptroller 2025 review of Local Law 144 compliance: DCWP's self-survey found 1 of 32 non-compliant; independent audit of the same companies found 17 potential violations. 17x is the real-world precedent; 3x is the trigger threshold, set conservatively.

**Method:**
1. Retrieve RAM's self-reported defect count for the claim class.
2. Count Evaluator defects of equivalent class.
3. If ratio > 3.0, fire the signal.
4. Attach absolute-count context; a 3x ratio on (1 vs 3) is noise-driven and is flagged as such.

**Severity:** high if absolute counts >= 5 on the independent side; medium if 2-4; flag-only with noise warning if 1.

**Ambiguity note:** co-emit to E9 regardless of absolute count; E9 externally evaluates signal vs noise.

---

## Signal 2 -- Missing demographic rows > 10%

**Trigger:** if the audit-bearing claim involves demographic distribution (fairness, disparate impact, representation) and more than 10% of the expected demographic rows are absent, imputed without disclosure, or labeled "test data."

**Empirical grounding:** Groene et al. / Sapiezynski et al. FAccT 2025 ("Auditing the Audits," N=116 Local Law 144 bias audits): most audits reported no four-fifths-rule violations; missing demographic data, "test data" substitutions, and opaque aggregation likely masked real violations.

**Method:**
1. Identify the demographic axes relevant to the claim.
2. Count expected rows (per the audit population definition).
3. Count actual rows with full demographic disclosure.
4. Compute missingness rate.
5. Fire if > 10%.

**Severity:** medium at 10-20%; high at 20-50%; critical above 50%.

---

## Signal 3 -- Fairness-metric choice not pre-registered

**Trigger:** the pre-registered fairness / audit criterion (see `PRE_REGISTRATION_HALT_GATE.md`) is absent, ambiguous on any of the three required components, or was modified after probes began.

**Empirical grounding:** COMPAS / Chouldechova -- same data yields opposite verdicts under different fairness definitions. Post-hoc selection is the canonical definition-dependent failure.

**Method:** evaluated at Phase A entry and re-checked via criterion-hash comparison before Phase F emits findings. Any hash mismatch across the run fires the signal.

**Severity:** always CRITICAL. Unique to Signal 3: firing HALTS the run.

---

## Signal 4 -- Test-data substitution for production data

**Trigger:** the RAM artifact's evidentiary basis is test / synthetic / sample data where production data is claimed or reasonably implied by the claim scope.

**Empirical grounding:** Groene / Sapiezynski field scan documented test-data substitution as a recurring mechanism masking production-level violations.

**Method:**
1. Read data provenance from the Audit Method Engine artifact.
2. Compare against claim scope; if claim scope implies production but provenance is test / synthetic, fire.
3. Also fire if provenance field is missing for a production-implicating claim.

**Severity:** medium if test-data use is disclosed but claim scope is ambiguous; high if claim scope is explicitly production and provenance is test; critical if provenance field is missing on a load-bearing production claim.

---

## Signal 5 -- Disclosure coverage << deployment coverage

**Trigger:** the disclosed scope (what was audited) is materially narrower than the deployed scope (what is in production use), and RAM's synthesis does not acknowledge the gap.

**Empirical grounding:** Local Law 144 pattern -- audits disclosed for ~2% of Fortune 500 employers; AEDT / ATS tools deployed by ~98%. Massive disclosure-vs-deployment gap.

**Method:**
1. Extract audited-population definition from the RAM artifact.
2. Extract deployed-population claim (explicit or implicit).
3. Compute coverage ratio = audited / deployed.
4. Fire if coverage ratio < 0.10 and gap is not disclosed; fire at any level if RAM's synthesis generalizes without acknowledging the gap.

**Severity:** medium if gap is < 10x; high if 10-50x; critical above 50x; always-critical if RAM's synthesis generalizes across the gap.

---

## Output

Every signal produces a record regardless of fire state (fire and null-fire both recorded) with schema:

```
{
  "signal_id": "SIG-01" | "SIG-02" | "SIG-03" | "SIG-04" | "SIG-05",
  "fired": bool,
  "severity": "low"|"medium"|"high"|"critical"|null,
  "evidence": { ... signal-specific ... },
  "run_id": <string>,
  "evaluated_at_phase": "A"|"C/D"
}
```

All records co-emitted to E9 for external over/under-report computation.
