Agentic coding tasks aligned to the Harbor evaluation framework across 10 languages (C, C++, Go, Rust, Python, Java, JavaScript, OCaml, Zig, Shell). Matched to Terminal-Bench 2.0, where Claude Opus 4.7 achieves 30.6% using the Terminus-2 harness (1,000 prompts, 5 rollouts per prompt). The dataset is composed of hard, medium, and easy tasks.
Available Now · Sample Available
Terminal-Bench
Agentic terminal coding tasks across 10 languages, aligned to Terminal-Bench 2.0.
Volume
0
Volume
Scalable Volume
Coverage
C, C++, Go, Rust, Python, Java, JavaScript, OCaml, Zig, Shell, Coding
Model Benchmark
0.0
%
Claude Opus 4.7 · Pass rate
Targets
Terminal-Bench 2
Modality & Delivery
TextCode→CodeText
Formats
Harbor-compatible task trees
What You Receive
Coverage
Covers systems programming, protocol implementations, parsers, and compiler concepts.
Package Structure
- <task_id>/One folder per task
- task.tomlHarbor task manifest u2014 metadata, timeouts, resource limits
- instruction.mdTerminal task description given to the agent
- environment/
- DockerfileReproducible base image with toolchain and repo snapshot
- repo/Target repository checked out at the base commit
- tests/Verification suite run against the agent's final state
- run_tests.shHarness entrypoint
- test_*.pyBehavioral checks (pass/fail)
- solution/Reference solution for harness validation
- solve.shOracle patch/script that makes all tests pass
Differentiators
- Harbor-aligned agentic coding tasks across ten programming languages.
Record Schema
Results
| Model | Metric | Result |
|---|---|---|
| Claude Opus 4.7 | Pass rate |
Methodology & Quality
Methodology
SME involvement
Domain-expert curated with up to L2 review
Stressors
Multi-language implementation in terminal-based environments.
Design criteria
- Cover systems programming, protocols, parsers, and compiler concepts across hard, medium, and easy tasks.
Build assumptions
Quality approach
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress