Chapter 19
Agent cost-per-task estimator
Estimate what one agent run costs in tokens: the fixed prompt, the history it re-reads every step, retries and subagents. Free AI agent cost estimator.
Home / Tools
Tools
Each tool computes one thing the book teaches, with the chapter beside it. They run entirely in your browser: nothing you type leaves the page, and every result can be copied as Markdown.
Chapter 2
Compounding error calculator
A free compounding error calculator for AI agents: whole-run success from per-step reliability, and the reliability a long task needs.
Chapter 12
Consequence tier classifier
Build a human in the loop approval policy for your AI agent: sort each action into four consequence tiers and get the gate each one needs. Try it free.
Chapter 7
Context window budget planner
A free context window budget planner: see how full your agent's window is, when it crosses the 40% rule of thumb, and which of four fixes to apply.
Chapter 16
Eval sample-size calculator
A free eval sample size calculator for AI agents: confidence intervals for a pass rate and the number of tasks you need, computed in your browser.
Chapter 16
LLM Judge Agreement Calculator
A free LLM judge agreement calculator: Cohen's kappa with an interval, precision and recall, the level check and bias tests. Paste your labels to start.
Chapter 17
Lethal trifecta audit
A lethal trifecta checklist for AI agents: see which legs your design holds, whether a sharp tool adds risk, and the cheapest leg to cut. Print the audit.
Chapter 16
pass@k and pass^k calculator
Compute pass@k and pass^k for an AI agent from its per-attempt success rate or your own runs, and see the reliability envelope. Free pass@k calculator.
Chapter 14
Should this be an agent?
When to use AI agents, and when plain code, one model call or a workflow does the job better. Answer ten questions and see where your task sits on the ladder.
Chapter 5
Tool schema linter
A free tool schema linter for LLM agents: paste a tool definition and get findings on its name, description, schema, side effects and hidden injections.