Checks your coding agent can't talk its way past.
deterministic-detector pairs same-turn lint feedback inside the agent's session with test-order randomization and mutation reporting in CI — on machines the agent doesn't control. Built for Python repos where an AI agent does real editing, and "the tests passed" can quietly mean "the tests passed in the one order they happen to be written in."
agent edits parser.py → hook fires before the turn can end E401 Multiple imports on one line F401 `os` imported but unused F401 `sys` imported but unused 1 file would be reformatted exit 2 — findings fed back to the agent, same turn agent fixes parser.py → hook re-fires exit 0 — clean
The core idea
Enforcement lives outside the agent boundary.
A check that runs inside an agent's own session is a check the agent can skip, reinterpret, or reason its way around. So the harness splits in two: fast feedback where the agent works, and every check that can actually fail a build on CI runners the agent has no access to — pinned to the exact commit under review.
Inside the session
fast, unskippable feedback — but never load-bearing
- ↺ruff hook fires after Claude edits or Codex patches a .py file and returns format and lint findings to the agent in the same turn. Silent in repos without ruff config.
- ↺smoke re-check re-verifies the hook after client updates, so a payload-schema change can't kill it silently.
Outside, in CI
deterministic verdicts on machines no agent controls
- ✓randomized-suite runs the full test suite in random order on every PR, seed preserved in the log for exact reproduction. Order-dependent failures are real bugs, surfaced instead of hidden.
- ✓mutation-report mutates only the non-test Python files changed against the base branch and lists every surviving mutant in the job summary — a weak test can't hide behind a green run.
- ✓tampercheck (since 0.4.0) reads each PR's diff and fails if the change weakened verification itself — deleted, skipped, or focused tests,
|| true, hollowed assertions. Installed pinned from PyPI, so a locally edited copy can't change what CI enforces.
Designed to work together
Deterministic evidence for the Dev Rigor gates.
These are separate tools with an explicit contract. Dev Rigor Stack Lite coordinates PLAN → BUILD → VERIFY → REVIEW → MERGE and sizes the work by blast radius. deterministic-detector supplies evidence the agent cannot replace with a confident explanation: immediate Ruff findings, randomized CI, and a changed-code mutation report.
Dev Rigor Stack Lite
Decides what proof the change needs.
Plans the unit, requires test-first implementation, challenges the claims, selects the review depth, and permits only a green-path merge. At a release boundary it adds the full gauntlet, documentation, newcomer, public-surface, and rollback gates.
deterministic-detector
Produces evidence for those decisions.
Returns Ruff findings in the editing turn, runs the test suite in randomized order on the exact PR commit, and reports surviving mutations in changed Python files. The stack consumes that evidence instead of treating a fixed-order local test run as sufficient.
Portable Agent Skills bundle for Codex, Claude Code, Antigravity, and compatible hosts. The stack still works without this detector; when both are installed, it consumes the detector evidence proportionally. Everything here also operates alone — or installs together, pinned and never mixed, as layer 3 of rigor-suite alongside tampercheck, whose CI lane this plugin wires per repo.
What's inside
Four pieces, one distribution unit.
The plugin ships the hook declaratively — no per-repo settings merges, nothing an agent can clobber. Everything a plugin can't ship (dev dependencies, CI wiring) is handled by the bundled installer skill, which recons the repo and reports before changing anything.
Same-turn ruff feedback
A stdlib-only Python script wired to PostToolUse. Reads Claude file paths
or Codex patch paths from the event payload, runs ruff format --check and ruff check,
and feeds failures straight back to the agent. Exits silently for non-Python files,
repos without ruff config, or missing ruff — it never breaks editing.
Claude Edit | Write | MultiEdit Codex Edit | Write | apply_patch ↓ shared Python hook ↓ ruff format --check ruff check
install-detector
Wires a target repo end to end: installs and pins pytest-randomly and
detect-test-pollution, instantiates the CI jobs, and proves each step with
verbatim evidence — no step reports PASS without its acceptance check having actually
run. smoke mode re-verifies the hook on demand.
Claude › /install-detector Codex › $install-detector recon env, tests, CI, ruff config randomize two runs, two seeds shown ci-jobs both informational at first graph repo indexed, one real query owner branch-protection recipe → you
Three CI lanes
GitHub Actions templates the installer adapts per repo. Diff-scoping fetches full history so neither diff-driven job can pass vacuously on an empty diff — they refuse loudly rather than silently producing no data. mutmut runs on Linux runners, where it actually works.
randomized-suite pytest -p randomly # seed kept in log mutation-report fetch-depth: 0 # no vacuous pass # survivors → job summary, exit 0 tampercheck git diff | uvx tampercheck==0.1.1 # weakened verification → exit 1
Local code graph
A pinned code-review-graph server runs through uvx and gives
either host structural queries for callers, dependencies, and affected tests. The graph
stays local in SQLite; the plugin does not upload repository code.
$ uvx --from code-review-graph==2.3.7 \ code-review-graph serve ✓ shared by Claude Code and Codex
Honest by design
What it deliberately is not.
Every constraint below came out of an externally audited design pass — the kind of thing most tooling decides silently. Here they're the product.
Equivalent mutants exist — mutations that provably can't change behavior, which no test can ever kill. A hard "zero survivors" gate fails legitimate code as a matter of course. The report still runs on every PR and still names what survived; it just can't fail your build on its own.
The hook runs after the edit lands (PostToolUse) —
it can't stop a bad write, and pre-checking content that doesn't exist yet is the wrong
shape anyway. What it guarantees is narrower and real: the agent cannot end a turn
without being confronted with the findings.
The required-status-check flag is where the whole architecture concentrates — one unticked box removes everything. So no agent ever touches branch protection, in either direction. The installer prints the one-command recipe and stops; the flip is yours.
A suite that has never run in random order probably carries latent pollution — that's the premise of the project. A day-one required check would block the first unrelated PR for a months-old bug: real bug, wrong hostage. The randomized job stays informational until five green runs across different seeds, then you flip it.
Quick start
Choose your host, restart, install into a repo.
01 · get the plugin
Install in Claude Code or Codex
Claude Code can load the repository directly. Codex installs the same repository as a marketplace and plugin.
git clone https://github.com/scottconverse/deterministic-detector \ ~/.claude/skills/deterministic-detector codex plugin marketplace add \ scottconverse/deterministic-detector codex plugin add \ deterministic-detector@deterministic-detector
For one-off testing:
claude --plugin-dir ./deterministic-detector. In the Codex desktop app,
you can instead install Deterministic Detector from the Plugins Directory. Review and
trust its Ruff hook, then start a new session.
02 · wire a repo
Run the installer where your agent works
In a new session, inside the target repository. It recons first, reports before changing anything, and ends with the owner recipe for branch protection.
Claude › /install-detector Codex › $install-detector either › run in smoke mode # after client updates
Needs: a Python repo
with pytest, ruff on PATH for the hook, uv for the local graph,
and GitHub Actions for the CI jobs.
Windows note: mutmut cannot run natively on Windows (upstream limitation, verified in source — an unconditional platform guard plus a POSIX-only stdlib import). That's fine by design: the load-bearing mutation lane is the Linux CI job, not your dev box.
Project status
Early-stage, self-hosted, and says so.
v0.4.0, packaged for Claude Code and Codex from one repository. The design went through two external audit rounds before any code was written, the harness was piloted on its own repository — this repo runs its own detectors on every pull request — and its detectors have now been exercised against a real, mature codebase they didn't grow up with. Every fact below is backed by the evidence reports in the repo, not by claims.
Verified
- ✓Hook proven red-first and end-to-end — violations → exit 2 with findings on stderr; clean, non-Python, no-config, and malformed-payload cases exit silently; confirmed firing on real agent edits in a live session.
- ✓Pilot completed on this very repository — installer acceptance, CI jobs on real PR SHAs, and the full weak-test demo: a green suite hid two surviving mutants, the job named them, strengthened tests killed 2/2. Burn-in: five green randomized runs across five distinct seeds.
- ✓One repository, two proven hosts — Claude Code's original package contract remains covered, while fresh Codex sessions installed the plugin, loaded its shared skill and instructions, returned Ruff findings from a real
apply_patch, and called the shared code-graph MCP server. - ✓Dev Rigor integration exercised end to end — the dual-host v0.3.0 unit flowed through plan, test-first build, adversarial verification, review, green PR checks, merge, Pages deployment, and a cache-busted live visitor audit, with both detector CI jobs supplying exact-commit evidence.
- ✓Four silent-failure classes caught and closed by the project's own make-it-fail-first rule — two before first release, two more found by the pilot itself (including baselines that fail before any mutant is evaluated, which previously read as clean).
- ✓Code-graph queries (v0.2.0) — an MCP-served graph (pinned, run via uvx, local SQLite index) answers callers and dependent tests as an enumerable set: validated 21/21 exact against grep ground truth, including a module-alias call that text matching misses. Run against a real, unrelated codebase at scale — a 66,963-edge index built in seconds — leaving that repository's working tree untouched.
- ✓Exercised beyond its own walls — the randomizer ran a mature, unrelated 1,983-test suite under three distinct seeds (all green) and the graph answered queries on its real index. The detectors run correctly on a codebase they didn't grow up with; that suite proved order-independent, so nothing pathological surfaced there.
- ✓tampercheck lane (0.4.0), dogfooded here — this repository's own pull requests now run the tampercheck verification-integrity gate, pinned from PyPI. The tool carries its own evidence: paired flag/clean fixtures with CI mutation proofs, and a false-positive rate measured on 1,331 real commits.
Not yet
- →An order-dependent failure caught in the wild — the randomizer catching real pollution is proven only by the pilot's deliberate demo; the one mature outside suite it has met so far was clean, so that catch hasn't happened on code no one planted it in.
The design record is public. The original task spec, the audited implementation plan (decision record D1–D9, risk register, phased acceptance criteria), and the build and pilot evidence reports — including every defect found and the measured costs — ship in the repository, unedited.
Get started
Give your agent reflexes it can't opt out of.
One repository for Claude Code and Codex, one skill to wire a repo. The only thing it asks of you is the one thing it refuses to automate: flipping the required check.