Advanced chemistry questions designed to challenge frontier reasoning models. Each record pairs a question with a worked solution and a final answer.
Available Now · Sample Available
Chemistry QSA
Advanced chemistry questions, pre-filtered to stump frontier models, from total synthesis to physical organic.
Volume
0
Volume
Scalable Volume
Coverage
Grad, PhD, Organic Chemistry
0.0
%
Grok 4.3
Targets
HLE, GPQA Diamond
Modality & Delivery
Text→Text
Formats
Excel (.xlsx)
What You Receive
Coverage
Questions span; total synthesis, transition-metal and photoredox catalysis, radical and pericyclic reactions, stereochemical analysis, conformational dynamics, spectroscopy, polymer chemistry, and physical organic chemistry.
Package Structure
Differentiators
- Every item is selected to stump Grok 4.3 Thinking in at least two of five attempts.
Record Schema
| Field | Description |
|---|---|
answer | The concise final answer associated with the question. |
question | The question or task presented in the dataset record. |
solution | The worked solution or supporting reasoning for the question. |
Results
| Model | Metric | Result |
|---|---|---|
| Grok 4.3 | Pass@1 | |
| Claude Opus 5 | Pass@1 | |
| Claude Opus 5 | Pass@3 |
Methodology & Quality
Methodology
Questions are expert-authored, then split across an SME pool for complexity validation before benchmarking.
SME involvement
Domain-expert curated with up to L2 review
Stressors
Design criteria
- Rigorous mechanistic or quantitative reference
- Questions are self contained
Build assumptions
Quality approach
- Difficulty screening uses repeated model attempts before an item is admitted.
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress