AI agent loops that finish, verify, and stop.
The code behind the daily Loop Engineering video series: reliability patterns for AI agents — implemented, commented for adaptation, and tested. Not a framework. Small, readable modules you copy into your project and own.
A new episode airs daily; its code lands in the library. The price rises as the collection grows — early buyers keep every update free.
Watch free. Run instantly. Own forever.
Watch
A new Loop Engineering episode airs daily on the AIS YouTube channel — one reliability pattern per day, free. The videos teach the why.
Run
Unzip the library and run any pattern instantly: pure Python 3.10+, zero dependencies, no API key. A deterministic mock model is included; every example ships with its tests.
Own
Plug your real LLM client into one file, copy the patterns into your project, and keep them forever. As episodes air, their code lands in your library at no extra cost.
What’s in the library
71 patterns — episodes 1–17 and 28–81, every Season 1 episode outside the Graph arc. The Graph arc (episodes 18–27) ships as the Graph Engineering library. Season 1 is 81 episodes.
Every example runs out of the box and ships with its own checks. The library about verification is itself verified — we run every test before every release.
# no install, no dependencies, no API key cd ep01-stop-conditions python stop_contract.py # watch the pattern run python test_stop_contract.py # prove it works on your machine # then plug YOUR model into one file (core/model.py) # and copy the pattern into your project
Stop contracts
The four-part contract (success check, retry budget, escalation, evidence) and a bounded runner that can never run forever or fail silently.
Maker / Checker
The producer never grades its own work; a separate checker returns evidence-anchored verdicts and focused repair packets.
State, not transcript
Truth lives on disk; kill the loop any time and resume from state, never from zero.
Worktree per agent
Parallel agents in isolated git worktrees: attributable, scope-checked, merge-gated.
The minimal agent loop
The smallest correct loop: the model is rented, the loop is owned, every exit bounded and evidenced.
Structured output
Schema-validated model output with precise, bounded repair when it doesn't conform.
Retry budgets
Three ceilings (attempts, tokens, seconds), checked before spending, with itemized bills on exhaustion.
Idempotent steps
Every side effect keyed and ledgered: run the whole job twice, nothing happens twice.
Smallest viable step
Stepwise loops that localize failure, keep progress, and make replanning free.
Failure classification
Five failure classes routed to five different recoveries with separate budgets; repeat wounds force a replan.
Idempotent tool calls
Operation keys embedded in the created object, read-act-verify against the remote system, duplicates escalated — a timeout is never proof the write failed.
Shadow mode
The L0-L4 autonomy ladder: decide everything, execute nothing, promote on arithmetic, and a kill switch that outranks the ladder.
Budget controller
Cheapest-capable model routing, progress signals per iteration, stop at the quality threshold even with budget left, replan on diminishing returns.
Typed Handoffs
Agents hand off typed, validated packets instead of prose: labeled facts, checkable plans, provenance, and repair at the source.
Skill Routing
Route only the versioned skills a task needs for each phase, reroute when the task grows, and require receipts showing the rules were applied.
Leases That Expire
Replace assigned_to with expiring leases: atomic claims, stale-writer tokens, progress-only heartbeats, and recovery packets that resume work.
Stop Certificates
Stop on a signed certificate (success, blocked, or futility) backed by outcome evidence, never on an agent's feeling that it's done.
Freeze the Verifier
Stop reward hacking: freeze and hash the acceptance contract, test it on a known-bad, and allow changes only through reviewed amendments.
Working Memory vs Event Log
Keep an append-only event log for audit and a compact working state for focus, so settled decisions stay settled and failed plans stay dead.
Scope Gates and Integration Branches
Prove parallel agents' work composes: scope checks from the real diff, stale-base rebases, cherry-picked integration branches, and conflicts resolved by intent.
Watermarks and Replay Windows
Exactly-once scheduled agents: overlap on purpose with a replay window, dedupe on stable IDs, and move the watermark only after the whole batch has durable receipts.
Write the Check First
The check defines the task: free verifiers first, an LLM judge only where no hard check exists, and no verifier means no loop, just hope.
Put Rules in Code
Prompts steer judgment, code enforces contracts: sanitizers and validators wrap the model so must-always rules hold every time instead of most of the time.
Evidence Logs for Agents
Log the decision, not just the action: per-task JSONL diaries with first-class verifications and a because-chain you can query and read at 3 AM.
Human Approval Gates
Gate what is irreversible, public or expensive; let drafts flow; bind approvals to reviewed evidence; and earn autonomy one action type at a time, all on an audit trail.
Complete Minimal Agent Loop
The foundations capstone: one research-and-report loop with a stop contract, maker/checker, state.json, a budget, a failure path, a human gate and a 12-point checklist.
Checker Rubrics That Fail Clearly
Binary yes/no criteria, each with an evidence rule and an exact fix, checked in parallel and versioned, so every failure maps straight to a repair.
Repair Packets
Repair with artifact + failed checks + evidence + fix only. Passed checks are never resent, each retry must shrink the failure list, and a growing list escalates.
LLM Judge Panels
A jury for high-stakes output: judges with different lenses, majority or unanimous gates, dissent treated as signal, an adversary trying to refute, cost set by stakes.
Fan-Out/Fan-In Without Chaos
Decompose into independent reads, give each its own context and contract, cap concurrency, cut slow tasks at a deadline, and verify the merge as its own task.
Pipelines Beat Barriers
Wait-for-all makes every item wait for the slowest. Let items flow through stages alone and add a barrier only for dedup, merge or compare stages.
Phase-Boundary Compression
Discover, plan, execute, verify. At every boundary the loop keeps the conclusions and drops the chatter, so each phase starts from a small, clean summary.
Crash-Proof Checkpoint Files
Save resumable state after every change, write it atomically, and resume from disk. A crash drill kills the loop at every write and checks that nothing is redone or sent twice.
Append-Only Run Ledger
One plain-text line per action (date, id, outcome). The loop checks the ledger before acting and appends after, which gives you dedup, an audit trail you can grep and memory between runs.
Eval Loop for Prompt Changes
Every prompt change is a deploy. Score it against golden examples, block any case that stops passing even when the average goes up, and add real failures as new goldens.
Honest Golden Sets
Real failures, edge cases, adversarial inputs and normal cases in honest proportion. Review the set like code: 60 stale happy paths prove nothing that 10 honest cases can't.
Self-Consistency Voting
Ask N times with different framings, then vote. Agreement earns confidence, a split triggers escalation, and independent checks reject unanimous answers that are wrong.
Escalation Tiers
Route loop failures by blast radius: quiet retries, logged skips, alerts that carry evidence and a proposed fix, and irreversible actions that wait for a human yes.
Small, Sharp Agent Tools
One tool, one job: strict schemas, error messages the agent can repair from, delete that previews by default, and output that admits what it left out.
Context Packing
Pack what matters: contract first, fresh single-copy evidence, rules repeated at the end, history left out, and tokens per task metered like cost.
Daily Loops You Can Trust
Scheduled agent loops that hold up with nobody watching: ledgered idempotent steps, alerts on every failure, drafts a human publishes, and trust earned by a clean week.
Watchdogs for Silent Failures
Catch the loop that silently stopped: heartbeat, freshness and sanity checks from outside, one alert per problem, and a watchdog kept simpler than the loop.
Research Pipeline Capstone
The patterns wired together: fan-out research, verified and judged claims, checkpointed state, an idempotent ledger, a watchdog, and a checklist the run has to pass.
Quarantine Gate
Fetched text is data: a read-only reader extracts cited facts, a deny-by-default gate binds each approval to the exact proposal, and the actor stays inside its own credential.
Sandbox Contract
Least privilege as code: read-only source, separate write areas, named commands and hosts, a short-lived scoped token, a budget that stops the run, and a sandbox that closes on exit.
Scope Contract
AGENT_TASK.md as a gate: allowed files, forbidden paths, path escapes rejected, case-exact matching, empty diffs refused, and attempts capped before review.
Secret Boundary
Redact before the prompt (including secrets cut at the length limit), let the model emit intent only, and let tools read scoped keys from the environment after their scope passes audit.
Output Gate
Model text is untrusted input: strict JSON, a narrow schema, bounds and path checks, encoding per destination, a short retry budget, and a fail-closed stop with hashed evidence.
Claim Ledger
Every material claim carries source, locator and quote; code re-derives it, negation and scope included, and only verified claims ship.
Cost Kill Switch
Budget each call before it runs, charge failed calls too, trip route breakers that re-open after a failed trial, alert early, and fire-drill the stops.
Spin Guard
Motion is not progress: repeated signatures and a progress window catch circles and alternating edit-and-test loops, then stop the plan and record what not to repeat.
Verifier Control
Treat the checker as fallible: retry infrastructure failures once, isolate artifacts, quarantine noisy checks with owners and review dates, and score checkers against goldens.
Model Version Gate
Pin exact snapshots with no aliases, prove upgrades on a golden set, canary by stable hash, and roll back on measured drift or missing metrics.
Door Redaction
Redact at ingress with stable typed tokens (emails, phones and the names you supply), scrub every artifact, and keep the token map separate with an expiry.
Tool Intake
Third-party tools as supply chain: inventory, exact pins, per-tool scope, documented tool-specific secrets, reviewed updates, and logs without prompt copies.
Rollback Contract
Reversible primitive, immutable snapshot, archive-first deletes, and a restore path that must pass a real drill before any unattended write.
Incident Playbook
Freeze first, preserve evidence append-only, scope damage from audit trails, roll back only by a tested runbook for the matching trigger, then harden.
Hardened Loop Checklist
An audit card where every PASS is earned by a probe, wrapped around a repair loop that stops on success, spin, budget, or step limit.
Control Plane
A deterministic queue and status table schedules bounded agents, verifies every output, and turns failures into repair tasks that carry the failure detail.
Model Routing
Route each step to the cheapest capable model, keep strong models for judgment, never truncate input to fit a budget, and prove every fallback in the logs.
Queue Flow
Pull ownership with leases, stale workers rejected, review gates for risky work, bounded retries, and dead letters instead of heroics.
Curated Durable Memory
One sourced, scoped fact per entry, verified before use, pruned when wrong, expired or unused, and never silently lost to a damaged file.
Freshness Contract
Date every source, expire caches, guard before generation, flag unsupported numbers, and publish what the answer is current as of.
Merge Train
Isolated worktrees feed a one-branch-at-a-time train: every merge gated, a missing gate result stops the train, and conflicts become attributed repair tasks.
Integration Gate
Compose locally green changes into one artifact, run the combined suite, rerun to catch flakiness, bisect real failures, and promote only what passed together.
Loop Health
One finish event per task, trended by loop and day from raw events, with drift alerts that wait for enough data.
Cost Ledger
Price the completed, verified unit with retries, failed work and human minutes folded in, then compare it with the measured manual baseline.
A/B Harness
Same frozen tasks, truly blind scoring, quality, cost and speed metrics, a significance test before shipping, and a note on why the loser lost.
Product Factory
Agent API output as a contracted product: verification against its own contract, run receipts, a bounded last-good fallback, and billing only for verified runs.
Fleet Dashboard
Every loop reports one validated row; verified needs real evidence, and publishing waits for a named human’s decision.
Blueprint Loop
Contract, checkers, owned state, enforced budgets and deadline, a human hold, trace, and stop reason: the whole season in one controlled loop.
Where it sits
Free demos & tutorials
- · Happy-path only — no stop conditions
- · No tests, no failure paths
- · Break the moment reality shows up
Loop Engineering Code Library
from $99 · one-time- · Reliability patterns, implemented + tested
- · Runs offline — no API key, no dependencies
- · Grows with the daily series; updates free
- · Model-agnostic: works with any LLM
Agent frameworks
- · Heavy abstractions you don’t control
- · Framework churn every quarter
- · The reliability layer is still on you
Get the library
$99 one-time · no subscription · covers your whole team
- ✓ 71 patterns from the series, implemented + tested
- ✓ Pure Python 3.10+ · zero dependencies · no API key
- ✓ Mock model included — swap in your real client in one file
- ✓ Free updates as new episodes air — buy once
- ✓ Email support at [email protected]
The honest deal: the price steps up as the library reaches milestones (15, 32, and the complete Season 1 release). Whatever you pay, you keep every future update free. Buying early is the discount. Digital downloads are final sale — watch any episode and run the free snippets first; what you see is exactly what you get.