Agent Experience

ref

A tiny CLI that verifies what an agent actually did, instead of trusting that it said "done."

You declare the expected result of an action before it happens. You (or an agent) do the action. Then ref reads the real state back, compares it to what you declared, and records a signed pass/fail. The check is a plain hash comparison — no model, no LLM, no network — so it can't be talked into lying.

Install

Requires bun. It's a single file.

# grab ref.ts into your repo, then:
alias ref="bun /path/to/ref.ts"
# or run directly
bun ref.ts <command>

Commands

command	what it does
ref intend <kind> <target> <expected> [note]	Record what you expect. kind is file, http, or cmd-stdout. expected is the exact content, or a sha256.
ref verify <target>	Read reality, compare to the intent, append a signed verdict. Exits 1 on FAIL.
ref gate <target>	Exit 0 if the last verdict passed, else 1. For scripts / CI.
ref log	Print the chained verdict ledger.

Example

# 1. say what the file should contain
ref intend file ./config.json '{"replicas":3}'

# 2. the agent writes it (correctly, or not)
echo '{"replicas":3}' > config.json

# 3. ref checks reality against your intent
ref verify ./config.json
✓ PASS  file ./config.json  (match)

# if the agent had written {"replicas":1} instead:
✗ FAIL  file ./config.json  (mismatch)   # exit 1 — blocked

What it checks

kind	reality it reads
file	sha256 of the file's bytes on disk
http	sha256 of status + body from a live GET
cmd-stdout	sha256 of a command's stdout

How it works

• The thing being judged (an agent) never runs the check. verify reads the target itself and hashes it — it ignores whatever the agent reported.
• A verdict is {expected, actual, pass} plus a sha256 of that record, chained to the previous verdict's hash (like a mini blockchain) and signed with an HMAC key. Editing a past verdict breaks the chain.
• No model is involved on purpose. A model can be argued into a wrong answer; sha256(reality) === expected cannot.

Use in CI / a script

ref intend http https://api.example.com/health "$EXPECTED"
deploy.sh
ref verify https://api.example.com/health # nonzero exit stops the pipeline

Limits

• It proves the result matches what you declared — not that what you declared was a good idea.
• The ledger lives in a local JSON file. Tampering is detectable (broken chain), not prevented. For a real trust boundary, run the verifier where the actor can't reach its key — see the capability-separated version in the repo.
• REF_KEY env sets the signing key; the default is a dev placeholder.