Overview Quickstart Guide Documentation & API Interactive Playground Research & Methodology
Interactive Simulator / Benchmark Explorer / In-Memory Execution

Interactive Testbed & Simulator

Step through genuine benchmark failure scenarios, inspect trial-by-trial Delta Debugging partitions, or test custom failure conditions in the client-side simulator.

Step through real agent failure scenarios.

Select a testbed from the catalog below. Use the playback controls to step through each binary partition trial or switch between raw terminal tape, formatted card, and JSON views.

scenario: currency_ast · seed: 42

Test custom directives with client-side ddmin.

Input candidate prompt instructions or configuration flags, define a trigger pattern that simulates an agent failure, and execute the Delta Debugging partition search directly in your browser.

ddmin Partition Trace & Result:
Click "Run Delta Debugging Search" to execute client-side partition isolation and counterfactual verification.

Simulator Mechanics

The client-side sandbox implements Andreas Zeller's standard ddmin algorithm. It recursively splits the candidate pool into 2 partitions, tests each subset against the trigger condition, increases granularity when subsets pass, and executes an empty set counterfactual check to verify 1-minimality.

Production Integration

In production, evaluations are not simple substring matches: your evaluator callable runs your agent harness, inspects tool call syntax, validates database queries, or queries mock APIs. Use the Python SDK (agentxray.diagnose) or explore the Hugging Face Space for full multi-mechanism evaluations.