Graduate engineering questions built for training on the kinds of applied reasoning assessed by MMMU and SciBench. Every item is stumped on ≥ 3 of 5 attempts; 26% are fully stumped (5/5). Each record pairs a question with a worked solution and a final answer.
Available Now · Sample Available
Engineering QSA
Graduate engineering across mechanical systems, electromagnetics, robotics, and optical hardware.
Volume
0
Volume
Scalable Volume
Coverage
Grad, PhD, Engineering
23.1%Grok 4.3 · Pass@1
Targets
MMMU (Engineering), SciBench, GPQA
Modality & Delivery
Text→Text
Formats
Excel (.xlsx)
What You Receive
Coverage
Mechanical systems, power and electromagnetics, control and robotics, and optical hardware. Subdomains include civil engineering, mechanical engineering, chemical engineering, electrical engineering, materials engineering, environmental engineering, industrial engineering, and computer science engineering.
Package Structure
Differentiators
- Structured question, worked solution, and final answer record format.
Record Schema
| A | B | C | |
|---|---|---|---|
| 1 | answer | question | solution |
| 2 | The concise final answer associated with the question. | The question or task presented in the dataset record. | The worked solution or supporting reasoning for the question. |
Results
| Model | Metric | Result |
|---|---|---|
| Grok 4.3 | Pass@1 |
Methodology & Quality
Methodology
Questions are expert-authored, then split across an SME pool for complexity validation before benchmarking.
SME involvement
Domain-expert curated with up to L2 review
Stressors
Design criteria
- Keep the question, supporting solution, and final answer as distinct record components.
Build assumptions
Quality approach
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress
Get Started
Request a Sample
Share your target domains and the OTS team will follow up with sample data and next steps.