AgentProbe

Introducing AgentProbe

AgentProbe is an independent publication about testing, debugging and understanding AI agents. Here is what it covers and how its articles are organised.

1 min read
On this page
  1. What you will find here
  2. How articles approach a problem
  3. What comes next

Many agent failures have parallels in ordinary software, but non-deterministic model output makes them harder to reproduce and explain: a run that worked yesterday loops today, a tool call returns something the model misreads, and the bill grows faster than the traffic. AgentProbe is a place to work through those problems carefully, for developers who build and operate agents.

What you will find here

The site has three sections:

  • Blog: technical articles on reliability, debugging, observability, evaluation, cost and architecture.
  • Lab: reproducible experiments, each to be published with the code and data needed to run it again. None are published yet.
  • Tools: free tools that run in your browser. The first, an agent cost calculator, is in development.

How articles approach a problem

Debugging articles here will be organised around one loop, because many agent failures yield to it:

The debugging loop

Reproducing a failure usually starts with the trace of a single run. A useful trace records, for each step, what went in, what came out and what it cost. A simplified, illustrative shape, not tied to any framework and not a production-ready tracing schema (a real one would also carry run and span identifiers, timestamps and redaction of sensitive data):

trace.ts
interface AgentStep {
  index: number;
  kind: "model" | "tool";
  name: string; // model id or tool name
  input: unknown;
  output: unknown;
  inputTokens?: number;
  outputTokens?: number;
  durationMs: number;
  error?: string;
}

With records like these, “the agent got stuck” becomes a question you can answer: which step repeated, with what input, and what the model saw before choosing it again.

What comes next

The first articles and the cost calculator are in progress. Subscribe to the RSS feed to follow along.