agentbench

# AgentBench for OpenClaw

Benchmark your OpenClaw agent's general capabilities across 40 real-world tasks spanning 7 domains.

## Commands

When the user says any of these, follow the corresponding instructions:

- **`/benchmark`** — Run the full benchmark suite (all 40 tasks)
- **`/benchmark --fast`** — Run only easy+medium tasks (19 tasks)
- **`/benchmark --suite <name>`** — Run one domain only
- **`/benchmark --task <id>`** — Run a single task
- **`/benchmark --strict`** — Tag results as externally verified scoring
- **`/benchmark-list`** — List all tasks grouped by domain
- **`/benchmark-results`** — Show results from previous runs
- **`/benchmark-compare`** — Compare two runs side-by-side

Flags are combinable: `/benchmark --fast --suite research`

## Running a Benchmark

### Step 1: Discover Tasks

Read task.yaml files from the `tasks/` directory in this skill:

```
tasks/{suite-name}/{task-name}/task.yaml
```

Each task.yaml contains: name, id, suite, difficulty, mode, user_message, input_files, expected_outputs, expected_metrics, scoring weights.

Filter by `--suite` or `--task` if specified. If `--fast` is set and `--task` is not, filter to only tasks where difficulty is "easy" or "medium".

Profile is "fast" if `--fast` was specified, otherwise "full".

List discovered tasks with count and suites.

### Step 2: Set Up Run Directory

Generate a run ID from the current timestamp: `YYYYMMDD-HHmmss`

Read `suite_version` from `skill.json` in this skill directory.

Create the results directory:
```
agentbench-results/{run-id}/
```

Announce: `Starting AgentBench run {run-id} | Profile: {profile} | Suite version: {suite_version} | Tasks: {count}`

### Step 3: Execute Each Task

For each task:

1. **Set up workspace**:
   - Create `/tmp/agentbench-task-{task-id}/` as workspace
   - Copy input files from `tasks/{suite}/{task}/inputs/` to the workspace (if inputs/ exists)
   - If the task directory contains a `setup.sh`: run `bash tasks/{suite}/{task}/setup.sh {wor...

# AgentBench for OpenClaw

Benchmark your OpenClaw agent's general capabilities across 40 real-world tasks spanning 7 domains.

Not a coding benchmark — tests file creation, research, data analysis, multi-step workflows, memory, error handling, and tool efficiency.

Same tasks and scoring as the [Claude Code version](https://github.com/agentbench/agentbench). Results are cross-platform comparable and submit to the same [leaderboard](https://www.agentbench.app/leaderboard).

## Install

Place this skill in your OpenClaw skills directory, or clone directly:

```bash
git clone https://github.com/agentbench/agentbench-openclaw.git ~/.openclaw/skills/agentbench
```

## Quick Start

```
/benchmark                              # Run all 40 tasks (full profile)
/benchmark --fast                       # Run 19 easy+medium tasks (fast profile)
/benchmark --suite research             # Run one domain
/benchmark --suite research --fast      # Run easy+medium in one domain
/benchmark --task research-summarize-doc # Run one task
/benchmark --strict                     # Tag as externally verified
```

## Domains

| Domain | Tasks | Difficulty | What It Tests |
|--------|-------|------------|---------------|
| File Creation | 9 | 2E, 3M, 4H | Documents, spreadsheets, project scaffolding, config migration, skill graphs |
| Research | 5 | 3M, 2H | Summarize, compare, multi-source synthesis, git archaeology |
| Data Analysis | 5 | 1E, 1M, 1H, 1X | Anomalies, statistics, multi-format reconciliation, log pattern detection |
| Multi-Step | 5 | 1M, 2H, 2X | Data pipelines, log analysis, repo refactoring, release preparation |
| Memory | 5 | 2M, 1H, 1X | Recall, constraints, context switching, progressive accumulation |
| Error Handling | 6 | 1E, 2M, 3H | Corrupted input, cascading failures, misleading errors, partial recovery |
| Tool Efficiency | 5 | 3E, 2H | Minimal reads, right tool choice, codebase navigation, targeted fixes |

*E=Easy, M=Medium, H=Hard, X=Expert*

## Scoring

Each t...