INTERNAL REMEDIATION AUDIT
TARGET: BLACK BOX RESEARCH LABS •
DATE: 2026-02-17 •
CLASSIFICATION: SELF-ASSESSMENT / PUBLIC
REPORT ID: BB-SELF-2026-001
STATUS: ACTIVE
VERIFIER: MIGUEL INGRAM
STATUS: ACTIVE
VERIFIER: MIGUEL INGRAM
DISCLAIMER: This is an internal self-assessment, not an independent third-party audit.
We apply the same forensic methodology to ourselves that we use on client engagements.
Every finding references real repository artifacts (commit SHAs where applicable) and real file paths.
The purpose is transparency, not certification.
EXECUTIVE SUMMARY
[ AUDIT SCOPE ]
Systematic forensic analysis of our own platform across five surfaces: security infrastructure, API endpoints, documentation integrity, audit engine correctness, and design system compliance. Sources include the Dec 2025 security incident response, TED-007 pre-sprint hardening analysis, an independent 8-point portfolio critique, and the Feb 2026 LLM alignment cross-reference.
Remediation Status
8
Findings Remediated
With commit-level evidence
0
Finding In Progress
Partially mitigated
8
Total Findings
Across 5 audit surfaces
Risk Assessment
MITIGATED
Security Posture
ALIGNED
Documentation Integrity
HARDENED
Engine Correctness
COMPLIANT
Design System
Audit Metrics
- • Verification Packets: 24 total across repo history
- • Audit Documents: 7 (technical audit, critique analysis, pre-sprint, remediation summary)
- • Atomic Commits (LLM Alignment): 7 commits, 6 files, 1 verification packet
- • Portfolio Findings: 42 across 4 client audits (all findingKind set)
FORENSIC EVIDENCE LOG
F-BB-SEC-001 — CRITICAL
Hardcoded GitHub Token in Git History
Kind: SECURITY_DEFECT
Status: REMEDIATED
Status: REMEDIATED
Location:
deploy-vercel-cli.sh:58 (deleted)
A GitHub personal access token was committed to the repository in a deployment script.
The token persisted in git history even after the file was deleted, creating a full account compromise vector
if the repository were ever made public.
Remediation:
1. Token removed from working tree (deploy script deleted)
2. BFG Repo-Cleaner scrubbed git history
3. Force push to origin (commit 619eac3)
4. New deploy-vercel-cli-secure.sh reads from env
5. Revocation/rotation and verification recorded in incident report Evidence:
docs/archive/AUDIT_REMEDIATION_SUMMARY.md and docs/archive/SECURITY_INCIDENT.md — incident timeline, remediation procedure, and verification status.
F-BB-SEC-002 — HIGH
Audit Pages Rendered Client Data Without Access Control
Kind: SECURITY_DEFECT
Status: REMEDIATED
Status: REMEDIATED
Location:
src/pages/audit/[id].astro, src/pages/audit/briefing/[id].astro
Audit target pages containing client names, repository URLs, and forensic findings were publicly accessible
to anyone who guessed or enumerated the URL path. The only protection was
noindex, nofollow
meta tags, which do not prevent direct access.
Remediation:
Implemented HMAC-SHA256 capability token gating. Routes now return
404 unless
a valid ?t=HMAC_SHA256(secret, id) token is present in the query string.
No expiry. Operator override via BB_ALLOW_TARGET_RENDER=true.
No index/listing/enumeration surface exists.
Evidence:
VERIFICATION_PACKET_TED-007-COMPANY_AUDITING_HARDENING.md — Claims 1-2, with Class C negative evidence (404 without token, 200 with token).
F-BB-INTEG-001 — HIGH
LLM Containment Claims Contradicted Operational Reality
Kind: PROCESS_SIGNAL
Status: REMEDIATED
Status: REMEDIATED
Location: 6 files —
security.astro,
DPA_TEMPLATE.md,
MSA_TEMPLATE.md,
humans.txt,
SPEC_PRE_LAUNCH.md,
Company_System_Dossier.md
Systematic cross-reference of public-facing security claims against operational protocols revealed
that 6 files made absolute claims about not using cloud-based LLMs.
In reality, Phase 4 (EXEC4) of the audit pipeline uses external LLMs for narrative synthesis
from extracted, PII-scrubbed metadata. The containment boundary is real (air-gapped static analysis),
but the absolute claims were false.
Remediation: 7 atomic commits with verification packet.
28b0ab4security.astro — truthful containment boundary language 46ac2f2DPA_TEMPLATE.md — aligned containment language c40d767MSA_TEMPLATE.md — aligned containment language 19ada68.husky/pre-commit — gating update for public/ surfaces 0e25854humans.txt — correct rendering mode description 1f3a25aSPEC_PRE_LAUNCH.md — correct rendering + isolation description 4d8d7ddCompany_System_Dossier.md — operational sovereignty alignment Evidence:
VERIFICATION_PACKET_SECURITY_PROTOCOL_LLM_ALIGNMENT.md — Full commit log with SHA-pinned evidence.
F-BB-PROCESS-001 — MEDIUM
Independent Critique: 6 Valid Findings from 8 Critiques
Kind: PROCESS_SIGNAL
Status: REMEDIATED
Status: REMEDIATED
Location:
docs/CRITIQUE_ANALYSIS_AND_DESIGN_FIXES.md
An independent agent review of all 4 published audit reports (CrewAI, Continue, HumanLayer, LangChain)
produced 8 critiques. We validated each against the codebase and accepted 6 as valid:
[VALID] C1: 2 of 4 telemetry findings were bare stat dumps with no interpretation
[VALID] C2: Kill zone aggregate finding misclassified as CODE_DEFECT (should be PROCESS_SIGNAL)
[INVALID] C3: CWE-78 gated framing — all claimed-missing content was already present
[VALID] C4: Approval regex could false-positive on negated phrases
[VALID] C5: Critical CVE (F-CO-9531) buried at position #7 of 8 findings
[VALID] C6: 42% of findings (13/31) missing findingKind taxonomy field
[VALID] C7: Portfolio depth inconsistency — LangChain only at SCAN depth
[VALID] C8: No forensic disclaimer or legal foundation documents
Remediation: All 6 valid critiques addressed.
findingKind made required in schema. LangChain upgraded from SCAN to STANDARD with 2 new Phase 3.6 trace findings.
Critical CVE reordered to top. Negation pre-filter added to approval regex.
Forensic disclaimer added to audit pages and briefing pages.
F-BB-ENGINE-001 — MEDIUM
Bug in Audit Engine's Own Bug-Fix Classifier
Kind: CODE_DEFECT
Status: REMEDIATED
Status: REMEDIATED
Location:
engine/analyzers/utils.ts — FALSE_POSITIVE_RE
The engine's
classifyBugFixCommits() function used a regex to exclude false positives
from bug-fix commit detection. The exclusion list included prefix|suffix|postfix,
which was both unnecessary (the primary regex uses \b word boundaries, so "prefix" never
matches \bfix\b) and harmful: real commits like "fix prefix handling" were incorrectly
excluded. Found via behavioral testing during self-audit.
Remediation:
Before: classifyBugFixCommits("abc123 fix prefix handling") → 0 (WRONG)
After: classifyBugFixCommits("abc123 fix prefix handling") → 1 (CORRECT) Evidence:
VERIFICATION_PACKET_BATCH_COMPANY_AUDIT_HARDENING.md — Claim 36, with behavioral test evidence (12 test cases).
F-BB-PROCESS-002 — MEDIUM
Post-Mortem: Commit Staged Only Packet, Not Actual Fix
Kind: PROCESS_SIGNAL
Status: REMEDIATED
Status: REMEDIATED
Location:
src/pages/security.astro — Commit 7e29a09
During a post-mortem forensic audit of the batch hardening work, systematic verification of all claimed completions
against codebase state revealed that commit
7e29a09 ("fix false NDA/MSA/E&O claims on security page")
modified only the AIV verification packet — the actual security.astro file had zero diff from main.
The fix was claimed as complete but was never applied. Caught by comparing git diff against the packet's claims.
Remediation: Applied the actual edit to
security.astro (conditional NDA/MSA/E&O language) in a subsequent atomic commit.
Evidence:
VERIFICATION_PACKET_BATCH_COMPANY_AUDIT_HARDENING.md — Claim 38 (RCA-1).
F-BB-SEC-003 — MEDIUM
API Endpoints Logged Raw PII and Disclosed Operational Status
Kind: SECURITY_DEFECT
Status: REMEDIATED
Status: REMEDIATED
Location:
src/pages/api/submit.ts,
src/pages/api/health.ts
The form submission endpoint logged raw email addresses, company names, IP addresses, and user agents
to server logs. The health endpoint performed outbound checks and disclosed dependency status
to any unauthenticated caller. Both violated the principle of minimal data exposure.
Remediation: PII logging removed from
/api/submit.
Health endpoint gated behind HEALTHCHECK_TOKEN — returns 404 without valid token.
Evidence:
VERIFICATION_PACKET_TED-007-COMPANY_AUDITING_HARDENING.md — Claims 3-4.
F-BB-DESIGN-001 — LOW
12 Design Specification Violations Across UI Components
Kind: PROCESS_SIGNAL
Status: REMEDIATED
Status: REMEDIATED
Location:
src/layouts/Layout.astro, src/pages/index.astro, global CSS
Systematic audit of every UI component against the "Forensic Minimalism" design specification
revealed 12 violations: wrong headline font (Newsreader instead of Playfair Display),
signal color misuse (yellow instead of #FF4F00), soft transitions instead of hard-cut interactions,
border-radius exceeding 2px maximum, external Google Fonts dependencies, and 6 others.
Remediation: All 12 violations corrected. Compliance verified across
typography (4-size hierarchy), color palette (signal <5% screen area, AAA contrast),
interactive states (hard-cut, zero transitions), grid system (12-column Swiss, 0.5px hairlines),
and anti-patterns (no gradients, no rounded corners, no scroll-jacking).
Evidence:
docs/archive/DESIGN_AUDIT_REPORT.md — Full 12-item audit with before/after for each violation.
REMEDIATION TIMELINE
PHASE 1 — EMERGENCY SECURITY
December 2025
Status: COMPLETE
- • P0 hardcoded token: revoked, BFG-scrubbed, force-pushed
- • CSP duplication resolved (single source in vercel.json)
- • 12 design spec violations corrected
- • Secure deployment script created
- • External critique validated (87% accuracy, 3 errors documented)
- • Hero section elevated from mid-level to principal quality
- • Incident response documentation completed
PHASE 2 — INFRASTRUCTURE HARDENING
January 2026
Status: COMPLETE
- • Token-gated audit routes (HMAC-SHA256)
- • PII logging eliminated from API endpoints
- • Health endpoint restricted behind token
- • Dependency install moved inside container
- • Hash-chained destruction ledger
- • AIV canonical block parsing hardened
- • SARIF output normalization
- • Plugin architecture for analysis tools
PHASE 3 — AUDIT QUALITY + DOCUMENTATION INTEGRITY
February 2026
Status: COMPLETE
- • Independent 8-point critique: 6/8 valid, all remediated
- • findingKind made required (was optional, 42% missing)
- • Phase 3.6 surgical traces added to LangChain
- • Phase 3.7 synthesis engine (4 rules) implemented
- • LLM containment claims aligned across 6 files
- • FALSE_POSITIVE_RE engine bug caught and fixed
- • Post-mortem caught phantom commit (RCA-1)
- • 87 claims tracked in batch hardening packet
PHASE 4 — FULL PIPELINE SELF-AUDIT
Planned
Status: PLANNED
Run the full EXEC pipeline (Phases 0-7) against this repository. Produce structured findings JSON via the
automated forensic factory. This curated assessment will be supplemented with quantitative static analysis,
dependency scanning, and churn-complexity intersection data.
VERIFICATION
Every finding in this audit references verifiable artifacts: commit SHAs, file paths, verification packets, and
documentation with line-level citations. The full evidence chain is stored in the repository under
.github/ (24 verification packets) and
docs/ (7 audit documents).
This audit follows the same AIV protocol we apply to client engagements. Each change is tracked via atomic commits paired with verification packets that document claims, evidence, and classification. The protocol is enforced by a pre-commit hook and validated by CI.
24
Verification Packets
87+
Tracked Claims (min.)
42
Portfolio Findings
This is what our methodology looks like applied to ourselves. Every client engagement produces artifacts of this caliber.
AUDIT CAPACITY IS STRICTLY LIMITED. VERIFY ELIGIBILITY.