# Your next user is an agent.

AJX is an open-source **skill + CLI** for agent usability testing.

Test how agents use your website, CLI, SDK, API, or developer tool. See what it
takes to complete a task and what gets in the way.

[Star on GitHub](https://github.com/royosherove/ajx).

## Get started

Choose a familiar tool in the website's prompt, then paste it into your agent.
Your agent installs AJX and prepares a small usability test. You don't need a
project of your own to try it. The default example below uses jq.

```text
Install the latest stable AJX CLI and skill by following
https://raw.githubusercontent.com/royosherove/ajx/main/docs/AGENT-QUICKSTART.md
Then load the installed AJX skill.

I want to use AJX to test jq.
Starting point: https://jqlang.org/manual/
Task: Filter a sample JSON inventory to in-stock items and export their names and prices as CSV.

Prepare any tools, generated sample data, and local test server needed in a temporary workspace.
Use the AJX skill to run one trial with a fresh agent, verify the result, and show me the report and journey.
```

The prompt's dropdown offers these examples. Selecting one changes the tool,
documentation link, and task together:

- [jq](https://jqlang.org/manual/): Filter a sample JSON inventory to in-stock items and export their names and prices as CSV.
- [curl](https://curl.se/docs/manpage.html): Download JSON from a local test server, save the response body, and make an HTTP 404 return a nonzero exit status.
- [SQLite](https://www.sqlite.org/cli.html): Import a sample orders CSV, calculate total spending per customer, and export the summary as CSV using the SQLite CLI.
- [Python Requests](https://requests.readthedocs.io/en/latest/user/quickstart/): Fetch JSON from a local test server with a timeout, handle HTTP errors, and save selected fields to a CSV using Python Requests.

1. **Choose an example.** Each task uses generated sample data in a temporary
   workspace.
2. **Check prerequisites.** macOS or Linux, Bash, curl, and internet access.
   A real test also needs an authenticated worker CLI and reporter.
3. **Read the results.** Open the report and visual journey to see the outcome,
   effort, and product improvements.

An agent receiving this page should read the [setup guide](/agents.md) before
installing or running a trial. It includes checks, conflict handling, skill
paths, and execution guidance.

### Manual installation

```sh
bash -o pipefail -c 'curl -fsSL https://github.com/royosherove/ajx/releases/latest/download/install.sh | bash' && export PATH="$HOME/.local/bin:$PATH"
```

The installer includes the native CLI and the matching complete Agent Skill
from the latest stable public release. No Go toolchain or GitHub login is
required. For an existing skill conflict, use the full replacement command in
the [quickstart](/agents.md); preserve customized files.

To explore the interface without model calls, follow the
[simulated demo instructions](/agents/#try-without-model-calls).

## How much time, how many tokens, how many retries?

Given a simple prompt and a link, what does an agent go through to finish a task
with your tool, website, or SDK? Let’s find out. AJX records the attempt, checks
the result, and shows where your product made the task harder.

1. **Give it a prompt and a link.** Describe a task, point AJX to your product or
   docs, and define what a successful result looks like. Start with a fresh worker.
2. **See what it takes.** Inspect elapsed time, retries, and token usage where
   the harness exposes it. Follow the journey to see where the agent changed
   course or got stuck.
3. **Fix one thing and test again.** Each product ask includes evidence and a
   way to check the change. Use a fresh attempt with comparable conditions.

## The console and reports

These are actual screenshots of AJX. The console was captured during its
simulated demo; the reports use fictional public example data. They do not
represent a real benchmark of a product or model.

- **Plan:** review the task, checks, worker, and access before starting.
  [Full screenshot](/screenshots/console-plan.webp).
- **Live:** inspect stages, available worker events, files, verification,
  and cleanup. Event detail depends on the harness.
  [Full screenshot](/screenshots/console-live.webp).
- **Report:** follow a proposed product change to its evidence.
  [Screenshot](/screenshots/report.webp) and
  [browsable report](/examples/runs/baseline-codex-r1/report.html).
- **Journey:** inspect forks, blocks, waits, and recoveries in order.
  [Screenshot](/screenshots/journey.webp) and
  [browsable journey](/examples/runs/baseline-codex-r1/pretty-journey.html).

[Screenshot provenance and dimensions](/screenshots/manifest.json).

### Explore the fictional example

The export-CLI example compares three conditions:

- Codex, product v1.2.0: task verified; three asks about a wrong documented flag,
  missing progress, and ambiguous command descriptions.
- Codex, product v1.3.0: task verified; one remaining progress ask.
- Kiro CLI, product v1.3.0, unattended: task not completed; one authorization ask.

The product versions and outcomes are invented, not AJX release versions or
measurements of model quality.

- [Full matrix](/examples/)
- [Matrix JSON](/examples/matrix.json)
- [Product asks as Markdown](/examples/runs/baseline-codex-r1/asks.md)
- [Journey as Markdown](/examples/runs/baseline-codex-r1/journey.md)
- [Events as JSON Lines](/examples/runs/baseline-codex-r1/events.jsonl)

## Compare the conditions your users depend on.

The same task can go differently with another harness, model, or environment.
Keep the task and success criteria consistent so the comparison is useful.

Supported worker adapters include **Codex, Claude Code, and Kiro CLI**. Other
Agent Skills hosts can coordinate using a supported worker adapter.

Use local or Docker execution and select the worker's skills. Record its
filesystem and network access. A fresh local folder is not a sandbox.

For an already briefed trial, `ajx ci` runs without an interview, applies
acceptance rules, and writes JSON and JUnit results. Worker authentication must
already be configured. Remote provisioning, remote shared dashboards, selected
plugins/hooks, baseline regression gates, and automatic improvement loops
remain designs.

## Instructions for agents

Use the quickstart for exact commands, installation recovery, and the path to a
first report. These resources need no JavaScript or browser automation:

- [Setup guide: HTML](/agents/)
- [Setup guide: Markdown](/agents.md)
- [Documentation index](/llms.txt)
- [Website and quickstart in one file](/llms-full.txt)
- [Structured content map](/site.json)
- [Public AJX source, MIT license, and documentation](https://github.com/royosherove/ajx)

## About Roy and more sites

[Contact Roy](https://forms.gle/jQSx5KoXvrVA1JCK7) ·
[@royosherove on Twitter](https://twitter.com/royosherove) ·
[About Roy](https://robotpaper.ai/about/)

- [Roy Osherove](https://osherove.com/)
- [Courses](https://courses.osherove.com/)
- [The Art of Unit Testing](https://artofunittesting.com/)
- [5 Whys](https://5whys.com/)
- [Elastic Leadership](https://elasticleadership.com/)
- [Pipeline Driven](https://pipelinedriven.org/)
- [RobotPaper.ai](https://robotpaper.ai/)
- [RobotPaper Podcasts](https://podcasts.robotpaper.ai/)