squash

AUDIT_BASELINE.md — Phase G Bulletproof Edition · Phase 1 Findings

Generated: 2026-05-01 Branch: claude/kind-tereshkova-b39079 Scope: Full repo audit against the 7-phase Bulletproof Plan, focused on the five pillars: determinism, cryptographic integrity, coverage, provenance, evidence-grade output. Method: static greps, radon cc -nc -a, pytest --collect-only, manual review of every file emitting a signature.

This document is the load-bearing baseline for v3.0.0. Every follow-up sprint references the line numbers and grades captured here. No fixes in this document — fixes happen in Phase 2+. Phase 1 is naming the disease.

0. Tooling baseline

Tool Status Notes
coverage (line + branch) ❌ not configured no .coveragerc, no [tool.coverage.*] in pyproject.toml, no CI step
mutmut ❌ not installed, never run mutation score = unknown / 0 measured
radon (cyclomatic complexity) ✅ now installed locally; not yet wired into CI average grade C (17.33) across 284 blocks — far above the Konjo bar
hypothesis (property-based) ❌ not in deps zero property tests today
atheris (fuzz) ❌ not in deps zero fuzz harnesses today
mypy --strict ❌ relaxed (ignore_missing_imports = true, no --strict) static type guarantees absent
bandit / semgrep / pip-audit / trivy / gitleaks / OSV-Scanner ❌ none configured zero pre-commit / CI security scanners
cyclonedx SBOM on release ❌ not produced dependency declared but unused for self-SBOM
slsa-github-generator ❌ not wired no SLSA L3 provenance on wheels / Docker / action
rfc8785 / JCS canonicalizer ❌ not in deps and not in code every signed payload uses json.dumps() variants
RFC 3161 trusted timestamping ❌ not integrated no TSA token in any cert
Sigstore / Rekor transparency log ❌ not integrated local-only “anchor” log in squash/anchor.py
Input manifest at ingest (SHA-256 every file before analysis) ❌ not produced scanners hash artifacts but no top-level manifest

Test count: pytest --collect-only -q → 5,226 tests collected (the master plan still cites 4,308 / 4,471 — number is stale, plan needs an update). Coverage and branch coverage of those 5,226 tests is unmeasured — Phase G Sprint 1.1 below establishes the baseline.

1. Determinism findings — every signing/attestation path that is not byte-identical today

A reproducibility test (run twice, diff SHA-256 of the canonical output) would currently fail on most of the modules below. The five-pillar promise is not yet honoured.

1.1 Canonical JSON violations (Pillar 1)

Every signed payload must be RFC 8785 (JCS) before hashing/signing. Today nothing uses RFC 8785; existing sort_keys=True, separators=(",", ":") is “close” but not equivalent (no NFC normalisation, no number canonicalisation, no escape-sequence normalisation).

File Line Violation Severity
squash/attest.py 405 json.dumps(obj, indent=2, default=str) — pretty-printed AND default=str silently coerces unknown types 🔴 critical — Tier 0 signing path
squash/attest.py 461 json.dumps(bom, indent=2) — no sort_keys, BOM written before signing 🔴 critical
squash/attest.py 619 json.dumps(parent_bom, indent=2) — same 🔴 critical
squash/slsa.py 141 json.dumps(statement, indent=2) — SLSA in-toto Statement written non-canonically 🔴 critical — provenance
squash/slsa.py 271 json.dumps(bom, indent=2) — same 🔴 critical
squash/anchor.py 118 json.dumps(value, sort_keys=True, separators=(",",":"), ensure_ascii=False) — close, but not RFC 8785; ensure_ascii=False invites homograph drift 🟡 elevated
squash/anchor.py 569, 603 log lines sort_keys=True only 🟡 elevated
squash/chain_attest.py 247, 250, 487 sort_keys=True only — composite chain digest 🟠 high (multi-component chain → drift compounds)
squash/hallucination_attest.py 815, 1048, 1077 sort_keys=True only on signed cert body 🟠 high
squash/drift_certificate.py 95, 231, 236, 487 same 🟠 high
squash/carbon_attest.py 856, 867 same 🟠 high
squash/carbon_attest.py 962 json.dumps(bom, indent=2, ensure_ascii=False) for written BOM 🟡 elevated

Resolution path (Phase 2): introduce squash/_canonical.py with a single canonical_bytes(obj) -> bytes that wraps rfc8785.dumps(); ban raw json.dumps in signing paths via a custom Semgrep rule (Phase 5).

1.2 Clock injection violations (Pillar 1)

Every attestation path that calls datetime.utcnow(), datetime.now(), or datetime.now(tz=...) inline is non-reproducible by construction. Each call must accept an injected clock: Callable[[], datetime] (default datetime.now(timezone.utc)).

File Line Violation Severity
squash/attest.py 481 datetime.now(timezone.utc).strftime(...) baked into attestation timestamp 🔴 critical — Tier 0
squash/hallucination_attest.py 1034 issued_at=datetime.now(tz=timezone.utc).isoformat() baked into signed cert 🔴 critical — Tier 1
squash/data_lineage.py 252 f"{model_id}{datetime.datetime.now().isoformat()}".encode() — clock value mixed into a hash input → cert ID changes every call → reproducibility impossible without fix 🔴 critical — repro killer
squash/webhook_delivery.py 441 datetime.datetime.utcnow().isoformat() + "Z" — Python 3.12 deprecates utcnow; also non-injectable 🟠 high
squash/oms_signer.py 233–241 datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%SZ") for filename stamping 🟡 elevated (filename only, not signed bytes — but still test-flaky)
squash/asset_registry.py 98, 360 datetime.datetime.now() (no tz) in non-attestation registry paths 🟢 nuisance (non-Tier-0/1, but also wrong: naive datetime)
squash/annual_review.py 298 datetime.datetime.now().year for default year 🟢 nuisance
squash/vex.py 848 (docstring) references datetime.utcnow() default — needs API audit 🟡 elevated

Resolution path (Phase 2): every Tier 0/1 module gains a clock: Callable[[], datetime] = lambda: datetime.now(timezone.utc) parameter; default-call sites pull from squash._clock.utc_now() which test fixtures monkeypatch.

1.3 UUID nondeterminism in signed paths (Pillar 1)

uuid.uuid4() in any field that becomes part of the signed body breaks reproducibility. Replace with uuid.uuid5(NAMESPACE, canonical_input) keyed on the canonical input bytes.

File Line Field Severity
squash/slsa.py 65, 122 invocation_id of SLSA BuildDefinition — inside the signed in-toto Statement 🔴 critical — provenance reproducibility
squash/hallucination_attest.py 1021 cert_id="hac-" + uuid.uuid4().hex[:16] — inside signed cert 🔴 critical
squash/carbon_attest.py 823 cert_id=f"carbon-{uuid.uuid4().hex[:16]}" — same 🔴 critical
squash/anchor.py 593 anchor_id="anc-" + uuid.uuid4().hex[:12] 🟠 high
squash/freeze.py 244 FREEZE-{uuid.uuid4().hex[:12]} 🟠 high
squash/approval_workflow.py 472, 544 appr-{uuid4}, rec-{uuid4} 🟠 high
squash/incident.py 188 INC-{uuid4} 🟠 high
squash/hallucination_monitor.py 367, 471 mon-{uuid4}, brc-{uuid4} 🟠 high
squash/vendor_registry.py 266 vendor_id from uuid4 🟡 elevated
squash/api.py 1380, 1409, 2747, 2800, 2862 API-layer record/job/event IDs (operational, not signed) 🟢 nuisance
squash/webhook_delivery.py 304, 380, 412 endpoint/payload/event IDs (operational) 🟢 nuisance

Resolution path (Phase 2): squash/_ids.py exposing cert_id(prefix: str, *, canonical_payload: bytes) -> str returning f"{prefix}-{uuid.uuid5(SQUASH_NS, canonical_payload).hex[:16]}". All Tier 0/1 IDs route through it.

1.4 Random / set / dict-order nondeterminism

Location Issue
squash/benchmark.py:430 list(set(model_ids))[:20] — set→list iteration order is platform-dependent before truncation
squash/license_conflict.py:875,942,969 seen: set[...] used as iterator — must sorted() before serialisation
squash/rag.py:159, squash/iso42001.py:414, squash/sarif.py:104, squash/hallucination_attest.py:693, squash/data_poison.py:1020, squash/board_report.py:437, squash/annual_review.py:320,469 set[...] defined; need audit of every downstream serialise
No random.seed() calls in any Tier 0/1 module today — but no random.* calls either, so this is currently safe. Phase 2 will gate any future use behind a seeded RNG injected through the signature path.  

Resolution path (Phase 2): add a Semgrep rule no-unsorted-set-serialisation that flags any json.*(... set ...) or for x in some_set: where the result is hashed.

1.5 default=str time-bomb (very subtle, currently latent)

squash/attest.py:405 uses json.dumps(..., default=str). Today every dataclass passed in has explicit __dict__/asdict conversion upstream, so the default= branch is never hit — but the moment a Path or a datetime slips into the dict, the serialised form changes shape silently and breaks reproducibility without raising. Replace with explicit canonical encoder that raises on unknown types.

2. Cryptographic integrity findings (Pillar 2)

Capability Present? Notes
Input manifest at ingest (SHA-256 every input file) ❌ scanners hash artifacts per-file but never emit a top-level input_manifest.json covering the entire input set of a run
Canonical signing payload (RFC 8785) ❌ see §1.1
Ed25519 signing ✅ (squash/oms_signer.py) underlying cryptography library used correctly; key management out-of-scope
RFC 3161 trusted timestamp ❌ no TSA integration; cert issued_at is self-asserted
Sigstore / Rekor inclusion proof ❌ squash/anchor.py is a local-only transparency log — not a public, append-only Sigstore Rekor entry
SLSA Build Level 3 provenance ❌ squash/slsa.py builds a self-asserted in-toto Statement; no slsa-github-generator workflow → not L3
squash self-verify CLI ❌ no command verifies squash’s own attestation chain

Resolution path (Phase 3): add squash/_input_manifest.py (called as the very first step of every CLI subcommand that ingests files), squash/_tsa.py (DigiCert / FreeTSA client), .github/workflows/release.yml calling slsa-framework/slsa-github-generator, and squash self-verify CLI command.

3. Coverage / mutation findings (Pillar 3)

Top-10 attestation-critical modules for first mutmut run (Phase 1.2 target)

Picked by: emits a signature, OR writes an attestation file, OR is loaded by every signed cert.

  1. squash/oms_signer.py (signing primitive)
  2. squash/anchor.py (transparency log + canonical JSON helper)
  3. squash/attest.py (root attestation pipeline)
  4. squash/slsa.py (SLSA provenance emitter)
  5. squash/chain_attest.py (composite cert)
  6. squash/hallucination_attest.py (signed cert)
  7. squash/drift_certificate.py (signed cert)
  8. squash/carbon_attest.py (signed cert)
  9. squash/data_lineage.py (LineageCertificate)
  10. squash/sbom_builder.py (BOM that gets signed)

4. Cyclomatic complexity findings (Pillar 5 → readability → audit-ability)

radon cc squash/ -nc -a → average C (17.33) — the floor for Konjo is B (≤10) average, max D per function. Current state has 60 functions ≥ D and 7 functions at F (≥30):

File Function Grade
squash/cli.py main (line 10609) F
squash/cli.py _cmd_github_app F
squash/cli.py _cmd_annex_iv_generate F
squash/cli.py _cmd_anchor F
squash/dashboard.py Dashboard.build F
squash/attest.py AttestPipeline F
squash/attest.py AttestPipeline.run F

squash/cli.py is 10,829 lines. Every _cmd_* should split into its own module under squash/cli/<cmd>.py. Phase 5 ticket.

squash/license_conflict.py (1,357 lines) — the master plan already calls this out; Phase 5 splits into database.py / scanner.py / reporter.py.

E-grade functions worth flagging (CC 21–30, prime mutation targets): InsuranceBuilder._profile_model, InsurancePackage.to_markdown, MunichReAdapter.format, WebhookHandler._handle_push, _render_1c, _cmd_attest_identity, _cmd_watch_regulatory, _cmd_deprecation_watch, _cmd_compliance_matrix, _cmd_vex, _cmd_drift_cert, _cmd_cloud_risk (×2), AnnexIVGenerator.generate, DataLineageTracer.trace, AttestationDiff.to_table.

5. Provenance findings (Pillar 4)

6. Evidence-grade output findings (Pillar 5)

Spot-checked five recent attestations (Annex IV, Bias Audit, Carbon, Hallucination, Drift Cert) — the human-readable reports cite figures and ratings, but none of the cited figures embed a content-addressed pointer (sha256:…) back to the input that produced them. Every claim must be a re-verifiable hash reference. Phase 4/5 deliverable: evidence_pointer = (claim_id, sha256, byte_range) schema applied to every numeric or rating field.

7. The 22 line-numbered fix list (drives Phase 2 sprint backlog)

Every line below becomes a unit-tested Phase 2 PR.

  1. squash/attest.py:405 — replace json.dumps(obj, indent=2, default=str) with canonical_bytes(obj); raise on unknown types.
  2. squash/attest.py:461 — same.
  3. squash/attest.py:481 — replace inline datetime.now(...) with injected clock().
  4. squash/attest.py:619 — same as #1.
  5. squash/slsa.py:65 — replace uuid.uuid4() default factory with uuid5 keyed on canonical input.
  6. squash/slsa.py:122 — same.
  7. squash/slsa.py:141 — replace json.dumps(..., indent=2) with canonical encoder before write/sign.
  8. squash/slsa.py:271 — same.
  9. squash/anchor.py:118 — upgrade in-process canonicaliser to RFC 8785 (rfc8785.dumps).
  10. squash/anchor.py:593 — uuid5 not uuid4.
  11. squash/chain_attest.py:247,250,487 — canonical encoder.
  12. squash/hallucination_attest.py:815,1048,1077 — canonical encoder.
  13. squash/hallucination_attest.py:1021 — uuid5 not uuid4.
  14. squash/hallucination_attest.py:1034 — injected clock.
  15. squash/drift_certificate.py:95,231,236,487 — canonical encoder.
  16. squash/carbon_attest.py:856,867,962 — canonical encoder.
  17. squash/carbon_attest.py:823 — uuid5 not uuid4.
  18. squash/data_lineage.py:252 — stop mixing wallclock into hash input; inject clock OR drop time from the hashed key.
  19. squash/webhook_delivery.py:441 — datetime.now(timezone.utc) (drop deprecated utcnow); inject clock.
  20. squash/freeze.py:244, squash/incident.py:188, squash/approval_workflow.py:472,544, squash/hallucination_monitor.py:367,471 — uuid5 not uuid4 (signed paths only).
  21. squash/oms_signer.py:233–241 — accept injected clock for filename stamping (test stability).
  22. Net-new: squash/_canonical.py, squash/_clock.py, squash/_ids.py, squash/_input_manifest.py, squash/_tsa.py, squash/_self_verify.py.

8. Phase 1 exit checklist (this sprint)

9. Konjo verdict

The repo today is functionally rich but evidentially weak: 5,226 tests of behaviour, zero tests of byte-identity; 60+ functions over the complexity threshold; a transparency log that only logs to itself; provenance that is self-asserted; and a signing pipeline that uses pretty-printed JSON with default=str four hundred lines into the most-trusted path in the codebase.

The fixes are surgical, not architectural — every nondeterminism site is one of three patterns (canonical JSON, injected clock, uuid5). Once Phase 2 lands, byte-identical attestations on rerun become the default, not a heroic exception. That is the unlock that makes Phase 3’s input manifest + RFC 3161 + SLSA L3 + Rekor anchor meaningful — there is no point timestamping or anchoring something that does not hash to the same value twice.

Make it konjo. The vessel is not yet seaworthy. We know exactly which planks need shaping.