---
name: blind-spot-mapper
description: Detects frame-level defects in RAM output (monoculture, covert tier-bias, forced closure, publication-bias miss, retraction miss, AI-contamination miss) under a pre-registered fairness / audit criterion. Runs a 5-signal integrity check, hosts seeded-blind-spot detection hooks for the Phase 9 calibration harness, and maintains a versioned blind-spot taxonomy across runs. Operates inside the Evaluator auditor system as component E5, consuming Audit Method Engine artifacts and emitting findings to Reopen Authority Enforcer (E7) and Self-Audit Resistance Monitor (E9).
---

# Blind-Spot Mapper -- Evaluator E5

## 1. Scope

### 1.1 In-scope
- Ingest pre-registered fairness / audit criterion BEFORE RAM run begins. This is LOAD-BEARING. No probe may execute until the criterion is registered, timestamped, and locked. See `references/PRE_REGISTRATION_HALT_GATE.md`.
- Ingest RAM output artifact (via Audit Method Engine) and run six frame-level defect probes:
  - Monoculture probe -- see `references/probes/MONOCULTURE_PROBE.md`.
  - Covert tier-bias probe -- see `references/probes/COVERT_TIER_BIAS_PROBE.md`.
  - Forced-closure probe -- see `references/probes/FORCED_CLOSURE_PROBE.md`.
  - Publication-bias miss probe -- see `references/probes/PUB_BIAS_MISS_PROBE.md`.
  - Retraction miss probe -- see `references/probes/RETRACTION_MISS_PROBE.md`.
  - AI-contamination miss probe -- see `references/probes/AI_CONTAMINATION_MISS_PROBE.md`.
- Execute 5-signal integrity inventory on every run -- see `references/FIVE_SIGNAL_INVENTORY.md`.
- Maintain versioned blind-spot taxonomy across runs -- see `references/TAXONOMY_VERSIONING_RULES.md` (rules) and `../_public/bias-taxonomy/BLIND_SPOT_TAXONOMY.md` (store).
- Host seeded-blind-spot detection hooks callable by Phase 9 calibration harness -- see `references/SEEDED_BLIND_SPOT_HARNESS.md`.
- Emit findings to Reopen Authority Enforcer (E7); co-emit diagnostic metadata to Self-Audit Resistance Monitor (E9).

### 1.2 Out-of-scope
- Re-research of RAM's claims (Audit Method Engine replication-mode responsibility).
- Three-axis confidence scoring (Independent Scoring Layer / E4).
- Runtime frame-divergence verdict (Frame Divergence Gate / E8).
- Build-time or runtime isolation verification (Isolation Verifier / E1).
- Scope-creep verdict (Audit Charter Lock / E6).
- Reopen trigger enforcement (Reopen Authority Enforcer / E7).
- Over-report / under-report / calibrated-silence determination (Self-Audit Resistance Monitor / E9).

### 1.3 Zero-overlap note
Blind-Spot Mapper is Evaluator-side. MUST NOT share library content, state, retrieval surfaces, or frame parameters with any RAM component. All writes flow through the Evaluator Search Ledger (owner: Independent Scoring Layer). No read from RAM's search ledger is permitted.

---

## 2. Classified Knowledge Inventory

### 2.1 Procedural knowledge (HOW)
- Six-probe sequencing under a loaded frame (fixed order; see Section 3.3).
- Signal inventory evaluation order -- pre-registration check first; all others after.
- Taxonomy update protocol -- when to increment version; how to register a new pattern.
- Hook interface for Phase 9 seeded-blind-spot injection.
- Output format for findings emitted to E7 and E9.

### 2.2 Declarative knowledge (WHAT)
- Defect-probe definitions -- one reference per probe in `references/probes/`.
- COMPAS / Chouldechova mathematical-incompatibility -- equal FPR/FNR and equal PPV cannot both hold when base rates differ; audit verdicts are definition-dependent. Captured in `references/PRE_REGISTRATION_HALT_GATE.md`.
- Local Law 144 empirical grounding -- ~2% Fortune 500 disclosure vs ~98% AEDT deployment. Captured in `references/FIVE_SIGNAL_INVENTORY.md` (Signal 5).
- NY State Comptroller 17x delta -- self-audit found 1 of 32 non-compliant; independent audit found 17. Captured in `references/FIVE_SIGNAL_INVENTORY.md` (Signal 1).
- Costanza-Chock et al. field scan -- first-party audits have best access; third-party audits lack access; no consensus on "findable" issue.
- Bandy typology -- over-report most visible in distortion audits; under-report most visible in narrow-demographic discrimination audits.

### 2.3 The 5-signal integrity inventory (LOAD-BEARING)
Full definitions in `references/FIVE_SIGNAL_INVENTORY.md`. Summary:

1. Self-audit vs independent-audit delta > 3x -- compare Evaluator finding count to RAM self-reported count; 17x is empirical precedent; 3x is trigger.
2. Missing demographic rows > 10% -- flag if audit-bearing demographic claim has >10% absent / imputed / test data.
3. Fairness-metric choice not pre-registered -- CRITICAL; HALT probes until re-registered.
4. Test-data substitution for production data -- flag when RAM artifact implies production but evidentiary basis is test/synthetic.
5. Disclosure coverage << deployment coverage -- flag when disclosed scope is materially narrower than deployed (Local Law 144 pattern).

### 2.4 Ambiguity register
- Primacy / recency hazard -- recent probe patterns dominate taxonomy additions over older patterns that remain live. Counter: fixed probe order AND fixed signal order (Section 3.3).
- Probe-narrowing under time pressure -- operator or upstream orchestration may pressure-reduce probe count. NEVER rule: no probe may be skipped; failed probes emit null-run record, not silent omission.
- Post-hoc fairness-criterion selection -- Chouldechova implies post-hoc selection is definition-dependent cherry-picking. Pre-registration gate in Section 3.1 structurally prevents this.
- Taxonomy version drift -- new defect patterns accumulate without version increments. Counter: MANDATORY version bump on any new pattern, signed by run ID.
- E5 x E9 masking -- over-report inflates E5's catch rate inside E5. Co-emit all findings to E9 so the Self-Audit Resistance Monitor computes over / under-report externally.

---

## 3. Procedure Skeleton

### 3.1 Phase A -- Pre-registered fairness / audit criterion intake (GATE)
Purpose: register the criterion BEFORE any probe runs. Read `references/PRE_REGISTRATION_HALT_GATE.md` for the full halt protocol.

Inputs: Audit Charter Lock output (scope + declared fairness / audit criterion); Frame Registry entry.

Steps:
1. Read the declared fairness / audit criterion from the Charter.
2. Verify the criterion names: (a) the fairness definition binding on this audit; (b) the threshold at which a violation is recognized; (c) the claim-type scope.
3. If any of (a) / (b) / (c) is missing or ambiguous, HALT and return CRITICAL finding to Audit Charter Lock (E6) with signal 3 fired. Do not proceed to probes.
4. Record the locked criterion in the Evaluator Search Ledger with run ID, timestamp, and criterion hash. Immutable for the remainder of the run.

Decision point: criterion registered and locked -> proceed to Phase B. Criterion missing or ambiguous -> HALT (do not probe).

### 3.2 Phase B -- RAM-output intake
Inputs: RAM output artifact (via Audit Method Engine); claim decomposition.

Steps:
1. Receive per-claim / per-source / per-methodology audit artifact.
2. Verify claim decomposition is complete -- no unclassified claims.
3. For each claim, determine applicable defect probes (probe selection per claim type, below).

### 3.3 Phase C -- Frame-level defect probes (FIXED ORDER)
Probe order is fixed to suppress primacy / recency hazard. ALL probes run. Null-run records mandatory; silent omission is a NEVER.

For each claim, execute in this order:

1. Monoculture probe -- `references/probes/MONOCULTURE_PROBE.md`.
2. Covert tier-bias probe -- `references/probes/COVERT_TIER_BIAS_PROBE.md`.
3. Forced-closure probe -- `references/probes/FORCED_CLOSURE_PROBE.md`.
4. Publication-bias miss probe -- `references/probes/PUB_BIAS_MISS_PROBE.md`.
5. Retraction miss probe -- `references/probes/RETRACTION_MISS_PROBE.md`.
6. AI-contamination miss probe -- `references/probes/AI_CONTAMINATION_MISS_PROBE.md`.

Each probe returns: `{fired: bool, severity: low|medium|high|critical, evidence: structured, probe_version: string}`. Record via `templates/PROBE_FINDING_RECORD.md`.

Probe selection per claim type:
- Quantitative empirical claim -- all six probes.
- Qualitative interpretive claim -- monoculture, covert tier-bias, forced closure, AI-contamination (retraction and pub-bias conditionally applicable).
- Methodological claim -- monoculture, covert tier-bias, forced closure.
- Synthesis / aggregation claim -- all six probes; monoculture and forced closure elevated scrutiny.

### 3.4 Phase D -- 5-signal integrity check
Run `references/FIVE_SIGNAL_INVENTORY.md` in fixed order. Any signal firing becomes a first-class finding.

### 3.5 Phase E -- Taxonomy update (versioning)
Rules in `references/TAXONOMY_VERSIONING_RULES.md`; store in `../_public/bias-taxonomy/BLIND_SPOT_TAXONOMY.md`.

Steps:
1. Compare each fired probe's evidence against current taxonomy version. Match -> increment occurrence counter.
2. No match -> register new taxonomy entry; INCREMENT semantic version (minor on new pattern under existing probe; major on pattern requiring new probe).
3. Emit taxonomy version to the Evaluator Search Ledger.
4. Emit taxonomy-version metadata to E9 so staleness can be audited across runs.

Decision point: new probe required -> flag to Phase 4 instruction engineering as reopen trigger. Probe addition is design-time; NEVER runtime.

### 3.6 Phase F -- Findings output
Outputs:
- Reopen Authority Enforcer (E7) -- `{run_id, claim_id, probes_fired, signals_fired, severity_max, taxonomy_version, ledger_refs}`.
- Self-Audit Resistance Monitor (E9) -- `{probes_run_count, probes_fired_count, nulls_emitted, criterion_hash, taxonomy_version, delta_metrics}`. REQUIRED so E9 externally computes over / under-report (prevents E5 x E9 masking).
- Evaluator Search Ledger (writer) -- every probe run (fired or null), every signal evaluation, criterion hash, taxonomy version. ZERO overlap with RAM's search ledger.

---

## 4. Anti-Deviation

### 4.1 Primacy / recency controls
- Pre-registration (Section 3.1) runs FIRST, unconditionally, before any probe. Primacy anchor; probe findings are recency stream. Non-negotiable.
- Probe order (Section 3.3) fixed across runs. Do not re-order based on perceived "likely defect."
- Signal order (Section 2.3) fixed. Signal 3 halts the run on fire; other signals are first-class findings but do not halt.

### 4.2 Anti-examples
- Post-hoc fairness-criterion selection -- Chouldechova's canonical definition-dependent failure. Any post-hoc selection forces full probe re-run under the newly registered criterion AND a reopen trigger.
- Probe-narrowing for throughput -- if a probe cannot run (e.g., Retraction Watch offline), emit null-run record with explicit unavailability reason.
- Catch-rate optimization -- do NOT tune probes to raise catch rate in isolation. Catch rate is computed by E9.
- Taxonomy-silent reclassification -- all taxonomy changes versioned and logged.
- Reading RAM's search ledger -- FORBIDDEN.

### 4.3 Anti-sycophancy NEVER rules
- NEVER narrow defect probes to reduce catch rate.
- NEVER suppress a fired probe to align with RAM's conclusion.
- NEVER emit "no findings" without explicit null-run records for every probe.
- NEVER modify the pre-registered criterion mid-run; criterion hash is immutable for the run.
- NEVER proceed without a locked criterion; HALT and return to E6.
- NEVER roll back a taxonomy version; new patterns are additive.

---

## 5. Failure Audit

| Failure mode | Signal | Response |
|---|---|---|
| Definition-dependent audit verdict (COMPAS pattern) | Pre-registered criterion absent, ambiguous, or modified mid-run | HALT; signal 3 fires; return CRITICAL finding to E6 |
| Catch rate inflated by over-report | E5 reports high catch while E9 precision falls | Flag E5 x E9 interaction to Phase 9; reopen E9 design per kill criterion |
| Taxonomy staleness | New pattern repeatedly slips; Phase 10 surfaces pattern not in taxonomy | Increment taxonomy version; flag for Phase 4 design review |
| Probe unavailability | Probe cannot execute | Null-run record with unavailability reason; flag to E9 as potential under-report risk |
| Signal 1 noise at low counts | Both RAM and Evaluator found few defects; 3x ratio is noise | Attach absolute-count context; E9 evaluates signal vs noise |
| Frame-level defect masked by convergence | E8 reports convergence after probes ran | E8 takes precedence; E5 findings quarantined until re-run under divergent frame |
| Probe-result leak into re-registration | Criterion modified after seeing probe results | Invalidate run; log CRITICAL; reopen trigger to E7 |
| Silent probe skip | No record; E9 reconciles expected vs emitted counts | E9 flags under-report mode; E5 design reopens |

Most fragile assumption: the pre-registered criterion can always be specified with sufficient precision before the run. If Phase 3 / Phase 4 surfaces claim types where pre-registration is structurally impossible, Blind-Spot Mapper design reopens.

---

## 6. Interfaces

| Interface | Direction | Contract |
|---|---|---|
| Audit Charter Lock (E6) | IN | Declared fairness / audit criterion, claim-type scope, charter hash |
| Audit Method Engine (E2) | IN | Per-claim / per-source / per-methodology audit artifact; claim decomposition |
| Frame Host (E3) | IN | Loaded Evaluator frame; retrieval hooks for Retraction Watch / Crossref / AI-contamination registries |
| Frame Registry (cross-cutting) | IN | Per-question frame pair; used to guard against post-convergence probes |
| Reopen Authority Enforcer (E7) | OUT | Findings record; severity; probe fire summary; signal fire summary |
| Self-Audit Resistance Monitor (E9) | OUT | Co-emitted diagnostic metadata; taxonomy-version metadata; null-run reconciliation counts |
| Evaluator Search Ledger (cross-cutting, owner E4) | OUT (write) | Every probe run (fired or null); criterion hash; taxonomy version; probe-evidence structured |
| Phase 9 calibration harness | IN / OUT | Seeded-blind-spot injection hooks; seeded-case IDs; detection-rate per seeded case |
| Blind-Spot Taxonomy store | IN / OUT | Versioned store; immutable history; additive updates only |

---

## 7. File Registry

Private (this skill only):
- `references/PRE_REGISTRATION_HALT_GATE.md`
- `references/FIVE_SIGNAL_INVENTORY.md`
- `references/TAXONOMY_VERSIONING_RULES.md`
- `references/SEEDED_BLIND_SPOT_HARNESS.md`
- `references/probes/MONOCULTURE_PROBE.md`
- `references/probes/COVERT_TIER_BIAS_PROBE.md`
- `references/probes/FORCED_CLOSURE_PROBE.md`
- `references/probes/PUB_BIAS_MISS_PROBE.md`
- `references/probes/RETRACTION_MISS_PROBE.md`
- `references/probes/AI_CONTAMINATION_MISS_PROBE.md`
- `templates/PROBE_FINDING_RECORD.md`

Public Evaluator (this skill contributes; E9 also reads):
- `../_public/bias-taxonomy/BLIND_SPOT_TAXONOMY.md`

---

## 8. Deliverables per run

1. Blind-spot findings -- structured, emitted to E7.
2. Diagnostic metadata co-emission -- structured, emitted to E9.
3. Evaluator Search Ledger writes -- every probe run and signal evaluation.
4. Taxonomy-version record -- per-run version stamp + any new-entry registrations.
5. Phase 9 harness response records -- seeded-case detection outcomes.
6. Null-run records -- explicit record for any probe that could not execute.

---

## 9. Kill Criteria (inherited)

- Phase 9 catch rate < 50% -> reopen Phase 2B E9; re-examine E5 probe-to-E9 interaction.
- Phase 9 catch rate >= 50% BUT precision < 50% -> over-report mode active; E5 probes flagged for over-specification review.
- Phase 10 adversarial surfaces new defect pattern not in taxonomy after >=2 runs without capture -> taxonomy-staleness reopen.
- Phase 4 cannot specify pre-registration gate with sufficient precision -> reopen E5 design.

---

## 10. Version

- SKILL version: 1.0
- Probe set version: 1.0 (six probes)
- Signal inventory version: 1.0 (five signals)
- Taxonomy store version: 1.0.0 (major.minor.patch; additive only)
