verification integrity, checked per diff
An AI coding agent told to “make the tests pass” has two options: fix the
code, or make the tests stop checking. tampercheck reads the diff and catches the second
one — deleted tests, .only, || true, hollowed assertions,
swallowed errors — before a green run stops meaning anything.
$ git diff | tampercheck tests/test_auth.py - assert login("u", "p") is True + assert True CRITICAL unconditional-success tests/test_auth.py:2 assert True why it matters: `assert True` can never fail; it stands where a real assertion should be. HIGH assertion-weakened tests/test_auth.py 1 assertion removed, 0 substantive added tampercheck: 2 finding(s) at or above --min-severity high $ echo $? 1
the problem
A linter checks the code that exists — it has no opinion about the test that was removed. CI reports green, because green is exactly what was engineered. The only defence left is a human reading every diff, which is the thing that stops happening as agent output scales.
It asks one question: did this change make the checking weaker than it was before? Every detection fires only on lines the change added — pre-existing conditions in a legacy codebase are never reported, which is what makes it survivable as a required CI check.
coverage
Each one catches a specific way a change can produce a green run
without earning it. Severity decides what fails the build — the default gate is
high.
| Detection | Severity | What it catches |
|---|---|---|
| test-deleted | critical | A test file removed by this change |
| test-focused | critical | A new .only / fit / fdescribe — silently excludes every other test while still reporting green |
| unconditional-success | critical | New || true on a test command, bare exit 0 in CI, process.exit(0) / sys.exit(0) in tests, assert True |
| test-skipped | high | A new unconditional .skip / .todo / xit / @pytest.mark.skip / #[ignore] |
| test-filtered | high | A new test-selection filter — --grep, pytest -k, jest -t, cargo test <name> — shrinking what runs without changing what's reported |
| assertion-weakened | high | A test file that removes materially more executable assertions than it adds — tests still run, still pass, just check less |
| swallowed-error | high | A new empty catch, broad except: pass, or empty Err(_) => {} arm |
| placeholder | medium | A new TODO / FIXME / not implemented / todo!() standing where behaviour should be |
workflow
git diff | tampercheck
tampercheck --from main --to HEAD
tampercheck --pr 123 # via gh
Local working tree, any two refs, or a GitHub pull request. Add
--json for a stable machine-readable schema.
# tampercheck: allow test_cart.py # superseded by test_cart_v2.py
Sometimes deleting a test is correct. Acknowledge it in the change itself —
visible to reviewers, unable to drift like an ignore-file. Findings print their
own remedy, and --allow KIND:PATH covers one-off runs.
The exit codes are the contract, and the tool ships a test that forces an internal failure to prove 2 stays distinct from 1. A verification tool that fails open manufactures false confidence.
Clean. Nothing at or above the failure threshold — the receipt that checking didn't get weaker.
Findings. The change weakened verification; fix it or justify it in the diff.
Tool failure. Bad input or git error — loudly distinct from a clean result, by contract and by test.
evidence, not vibes
A checker earns trust by staying quiet on honest work. The patterns were run against the full recent history of three actively developed repositories and tightened until ordinary changes stopped lighting it up.
Every detector has a paired fixture — a diff that must be flagged and a near-identical clean twin that must not be — and CI demonstrates each one observably firing before it is trusted. A detector that has never been seen going red is not proven to work. Tuning decisions are recorded next to the patterns they shaped.
honesty
tampercheck is a line-oriented diff check. It does not run your tests and does not judge code quality. It will not catch:
It reduces the cheapest forms of verification tampering to zero cost for a reviewer. It does not replace the reviewer.
works with
tampercheck stands alone — any repo, any CI, any author. It also slots into a layered stack for keeping AI-agent work honest.
Evidence-first agent skills whose verify and merge gates cite tampercheck receipts when the tool is present.
layer 3 · enforcementWires a pinned tampercheck lane into a repo's CI with /install-detector — on a machine no agent controls.
One command installs all three layers, pinned, without mixing them.
quick start
Zero-install via uv, or install normally. Then pipe it any diff.
uvx tampercheck --version
# or
pip install tampercheck
git diff | tampercheck
Pin the version in CI so a locally edited copy can never change what's enforced.
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- run: |
git diff origin/main...HEAD \
| uvx tampercheck==0.1.1