Sensors (deterministic feedback)

Sensors are the feedback control: they observe the artifact after the agent writes it and return a hard pass/fail. A failing sensor blocks approval and the agent regenerates until it passes.

Every sensor declares how it is enforced, and the distinction is the whole point:

This is Böckeler's computational/inferential split, and harness-kit learned it the hard way. Three sensors once declared themselves deterministic hard gates while writing their checks as prose the runner had no handler for. The runner returned 0, and the quality log recorded them as passed on every run, against a check that never happened. A sensor that cannot report a pass it did not earn is worth more than one that always says yes.

Run python3 .claude/scripts/check-sensors.py to print the enforcement ledger: every sensor, its execution type, and the checks it actually wires up.

What a sensor is

A sensor is a plain markdown file describing checks, plus a small Python runner that applies them with real regex. The markdown is the spec; the runner is the enforcement. Example shape:

# Sensor: PRD Structure
Type: deterministic
Execution: computational
Mode: hard gate

## Required sections
- Problem and Hypothesis
- Customers
- ...

## Forbidden tokens
- lorem, TODO, FIXME, placeholder

## Markdown rules
- exactly 1 H1 heading
- no em-dash
- no ASCII box-drawing

## On failure
Block publish. Return missing sections, forbidden tokens, rule violations.
Agent regenerates failed parts only.

How the runner works

sensor-runner.py parses known sections out of the sensor markdown and checks them against the artifact:

• Required sections: for each bullet, it builds a heading regex (tolerant of numbered prefixes like ## 3) ...) and fails if the heading is absent.
• Forbidden tokens: fails if any appear (catches TODO, lorem, unfilled template tokens like {System Name}).
• Markdown rules: exactly one H1, no em-dash, no ASCII box-drawing, minimum mermaid blocks, etc.

Exit codes carry the distinction:

Exit 2 exists because the alternative is silence. A sensor whose checks the runner cannot parse used to return 0 forever, which reads as "checked and fine" when the truth is "never looked". A PostToolUse hook fires the runner on save and returns the feedback to the agent.

Heading matching is lenient on purpose: a trailing parenthetical is ignored and a descriptive prefix is allowed, so ## Design doc, required sections (all present, in order) resolves to Required sections. The original bug was exactly this: the runner accepted only (all must be present, in order), three sensors wrote (all present, in order), and about 30 written section assertions did nothing.

Why deterministic

Structure is not a matter of opinion, so it should not cost an LLM call or be subject to a model's mood. Pushing every checkable rule into a sensor:

• makes the gate fast and free (no tokens),
• makes it exact (no false "looks fine"),
• frees the eval to judge only what genuinely needs semantic judgment.

This is the core harness-engineering move: make deterministic what you can, infer only what you must.

Sensors in harness-kit

Each stage has its own sensors. A sample:

code-maintainability is the one that leaves the document and looks at the code. It runs whatever the repo actually configures (npm lint/typecheck, ruff, ktlint, checkstyle, gitleaks) and never an imposed config. When a repo configures nothing it knows, it exits 4, not checked, rather than passing. Nothing ran, so nothing is known.

design-rigor is the honest counter-example. It wants numeric targets, visible back-of-envelope math, at least three trade-offs and a vertical-slice phase. Those need judgment, so it is inferential: a model applies it and the log says so. It spent a long time claiming to be a deterministic hard gate while checking nothing at all.

Feedback optimized for the agent

A sensor's value is not just the verdict, it is the message. "missing required section: 'Success Metrics'" tells the agent exactly what to add. Good sensor output reads like instructions for the next turn, not a stack trace. This is what makes deterministic feedback a self-correction loop instead of a wall.

Writing a sensor

• Declare Execution: honestly. If the runner cannot check it, it is inferential, and saying so costs you nothing. Claiming computational and expressing the check as prose is how you get a sensor that reports green forever without looking at anything.
• Express computational checks in a section the runner parses: Required sections, Forbidden sections, Required tokens, Forbidden tokens, Markdown rules. Anything else is documentation, not enforcement.
• Keep checks objective: if a human could disagree, it belongs in an eval or an inferential sensor.
• Name the failure precisely so the fix is obvious.
• Use forbidden tokens to catch unfilled template placeholders ({N}, {...}, TBD).
• Mark it hard gate; sensors do not have a threshold, they pass or block.
• Run python3 .claude/scripts/check-sensors.py before you commit it. CI runs it too, and the test suite asserts that every computational sensor rejects an empty artifact, so a sensor that checks nothing fails the build instead of shipping.

See also

• Evals: the inferential half that judges meaning
• Harness Engineering: computational vs inferential controls
• References: the literature this is built on
• Böckeler, Maintainability sensors for coding agents, martinfowler.com. The source of the computational/inferential split, the maintainability sensor, and the warning about the "illusion of quality" that a sensor checking nothing produces.

This page mirrors Sensors (deterministic feedback) in the wiki. Edit it there.