Instruction-following prompts across varied fields, each paired with 25 test cases that check whether a model satisfies every instruction across different scenarios. Opus 4.5 scores only 20.79%.
Available Now · Sample Available
Complex Instruction Following
Complex multi-part instructions that frontier models still fail to fully satisfy.
Volume
0
Volume
Scalable Volume
Coverage
Instruction Following
20.8%Claude Opus 4.5 · Pass rate
Targets
IFEval, IFBench
Modality & Delivery
Text→Text
Formats
JSON
What You Receive
Coverage
Package Structure
Differentiators
- Each prompt includes twenty-five test cases that check whether all instructions were followed.
Record Schema
constraintsarray<string>The individual atomic constraints decomposed from the instruction, used for programmatic verification.
instructionstringThe complex multi-part instruction the model must follow.
reference_responsestringAn expert-written response that satisfies every constraint.
rubricstringScoring guidance used to grade partial constraint satisfaction.
Results
| Model | Metric | Result |
|---|---|---|
| Claude Opus 4.5 | Pass rate |
Methodology & Quality
Methodology
SME involvement
Domain-expert curated with up to L2 review
Stressors
Prompts with many simultaneous constraints across varied domains.
Design criteria
- Score instruction compliance at the requirement level rather than by a single holistic judgment.
Build assumptions
Quality approach
- Twenty-five explicit test cases per prompt.
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress
Get Started
Request a Sample
Share your target domains and the OTS team will follow up with sample data and next steps.