Instruction-following prompts across varied fields, each paired with 25 test cases that check whether a model satisfies every instruction across different scenarios. Opus 4.5 scores only 20.79%.
Available Now · Sample Available
Complex Instruction Following
Complex multi-part instructions that frontier models still fail to fully satisfy.
Volume
0
Volume
Scalable Volume
Coverage
Instruction Following
Model Benchmark
0.0
%
Claude Opus 4.5 · Pass rate
Targets
IFEval, IFBench
Modality & Delivery
Text→Text
Formats
JSON
What You Receive
Coverage
Package Structure
Differentiators
- Each prompt includes twenty-five test cases that check whether all instructions were followed.
Record Schema
| Field | Description |
|---|---|
constraints | The individual atomic constraints decomposed from the instruction, used for programmatic verification. |
instruction | The complex multi-part instruction the model must follow. |
reference_response | An expert-written response that satisfies every constraint. |
rubric | Scoring guidance used to grade partial constraint satisfaction. |
Results
| Model | Metric | Result |
|---|---|---|
| Claude Opus 4.5 | Pass rate |
Methodology & Quality
Methodology
SME involvement
Domain-expert curated with up to L2 review
Stressors
Prompts with many simultaneous constraints across varied domains.
Design criteria
- Score instruction compliance at the requirement level rather than by a single holistic judgment.
Build assumptions
Quality approach
- Twenty-five explicit test cases per prompt.
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress