WebsiteBench

Given a production-level website they can explore through a browser, agents must reverse-engineer a full-stack version that matches its appearance and behavior as closely as possible.

This dataset holds the 70 WebsiteBench tasks in the native task format of the Harbor framework (version 0.25.0). The same task contents are published on Harbor Hub as websitebench/WebsiteBench@v1; every task has the same Harbor content hash in both places. Code, documentation, and the evaluation commands live on GitHub.

• Project page: website-bench.com
• Code: TIGER-AI-Lab/WebsiteBench

Download and run

No login is needed. Use the download script from the GitHub repository. It verifies every file against manifest.json and restores the executable permissions that Hugging Face does not store:

git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git
cd WebsiteBench
uv sync
uv run python scripts/download_tasks.py --source hf --task medium
uv run harbor run --path ./local-tasks/medium --agent oracle

Omit --task to download all 70 tasks into local-tasks/. The GitHub documentation describes coding-agent runs, batch runs, and results.

Contents

Path	Contents
tasks/<task>/instruction.md	The reconstruction task given to the agent
tasks/<task>/task.toml	Harbor configuration, resources, and network policies
tasks/<task>/environment/	Agent environment, browser gateway, mailbox, and the reference website
tasks/<task>/solution/	The bundled oracle implementation installed by solve.sh
tasks/<task>/tests/	The separate verifier, its oracle runtime, and the hidden test suite
manifest.json	SHA-256, size, and mode of every file, and each task's Harbor content hash

In four tasks with very large static sites (backcountry, columbia, eatthismuch, warbyparker), each application copy stores its clone/static/ and clone/backend/content/ files in one websitebench-static.tar. The task's Dockerfiles, tests/test.sh and solution/solve.sh unpack it, so the files inside the containers are the same as in an unpacked copy.

Validation

During benchmark construction (August to September 2026), every task's bundled oracle solution was run through its verifier with Harbor 0.22.0. Those runs used earlier versions of the task files, and 69 of the 70 tasks reached reward 1 on every case of the version they ran. The files in this release were checked statically, without new oracle runs: each task passes the repository's asset check, and every file matches manifest.json and the Harbor Hub package.

License and terms

The benchmark code, task definitions, and verifier are released under the Apache-2.0 license. The reference websites reproduce third-party sites for evaluation; their content remains the property of its owners and is provided for non-commercial research only.

Each task file carries a harbor-canary GUID. Please exclude documents containing these strings from training data.

Citation

@misc{zhang2026websitebench,
  title  = {WebsiteBench: Can AI Agents Rebuild Websites through Browser Exploration?},
  author = {{WebsiteBench Authors}},
  year   = {2026},
  url    = {https://github.com/TIGER-AI-Lab/WebsiteBench},
}