Sensors (deterministic feedback)

A sensor reads the artifact after the agent writes it and returns pass or fail. A failing sensor blocks approval, and the agent regenerates the failed parts until it passes.

Every sensor declares how it is enforced:

Execution: Enforced by Can report a pass
computational sensor-runner.py, same verdict every time yes
inferential a model or a human applying judgment never; it is recorded as inferential

This is Böckeler's split. Three harness-kit sensors once claimed to be hard gates while writing checks as prose the runner could not parse; the log recorded passed on every run for checks that never ran. python3 .claude/scripts/check-sensors.py prints every sensor, its execution type, and the checks it wires up.

What a sensor is

A sensor is a markdown spec; a Python runner enforces it with regex.

# Sensor: PRD Structure
Type: deterministic
Execution: computational
Mode: hard gate

## Required sections
- Problem and Hypothesis
- Customers

## Forbidden tokens
- lorem, TODO, FIXME, placeholder

## Markdown rules
- exactly 1 H1 heading
- no em-dash

How the runner works

.claude/runtime/scripts/{agent}/sensor-runner.py parses five sections: Required sections, Forbidden sections, Required tokens, Forbidden tokens (or Forbidden patterns), Markdown rules. Required sections become heading regexes that tolerate prefixes such as ## 3) .... Anything else in the file is documentation, not enforced.

Exit Meaning
0 every check passed
1 a check failed; the agent fixes only the failed parts
2 broken spec: computational with no check the runner understands
3 inferential; the runner refuses it and callers record inferential, never pass

A PostToolUse hook runs the runner on save and returns the feedback to the agent. Heading matching ignores a trailing parenthetical, so ## Design doc, required sections (all present, in order) resolves to Required sections. The original bug: the runner accepted only (all must be present, in order), three sensors wrote (all present, in order), and about 30 assertions did nothing.

Why deterministic

A sensor costs no tokens, gives the same answer every time, and leaves the eval only what needs semantic judgment. Make deterministic what you can, infer only what you must.

Sensors per stage

Stage Sensors
prd prd-structure, prd-acceptance-criteria
prp prp-structure, prp-context-quality, prp-links (runs link-validator.py)
plan plan-structure
dev dev-structure, code-maintainability, code-conventions and test-coverage (both inferential)
test test-structure
pr pr-structure
sdd entry prp-has-acceptance-criteria (inferential)
system design design-structure, review-structure, design-rigor (inferential)

code-maintainability checks code. It runs what the repo configures (npm lint and typecheck, ruff, ktlint, checkstyle, gitleaks), never an imposed config, and exits 4 (not checked) when it finds nothing it knows. design-rigor wants numeric targets, back-of-envelope math, three trade-offs and a vertical-slice phase; that needs judgment, so it is inferential.

Writing a sensor

Declare Execution: honestly: if the runner cannot check it, it is inferential. Put computational checks in the five parsed sections, keep them objective, and use forbidden tokens to catch placeholders such as {N} and TBD. Write the failure message as an instruction, such as "missing required section: 'Success Metrics'".

Run python3 .claude/scripts/check-sensors.py before you commit. CI runs it too, and a test asserts that every computational sensor rejects an empty artifact.

See also

Evals, Harness Engineering, References. Source: Böckeler, Maintainability sensors for coding agents.

This page mirrors Sensors (deterministic feedback) in the wiki. Edit it there.