Multimodal prompts and scoring rubrics authored by native language experts, evaluating whether models reason and respond appropriately in realistic, culturally rich scenarios across English, Korean, Arabic, Japanese, Spanish, Chinese, German, and Brazilian Portuguese. GPT-5.2 scores only 12.4%.
Available Now · Sample Available
Cultural Alignment Dataset
Native-expert prompts and rubrics testing implicit cultural understanding across eight languages.
Volume
0
Volume
Scalable Volume
Coverage
Cultural Alignment
12.4%GPT-5.2 · Pass@1
Targets
CulturalBench, BLEnD, Global-MMLU
Modality & Delivery
Text→Text
Formats
JSON
What You Receive
Coverage
Implicit cultural context across languages, locales, and multimodal tasks — going beyond trivia-style recall. Findings show that models consistently struggle more with implicit cultural knowledge than explicit; models perform roughly 2x as poorly when cultural requirements must be inferred from context rather than stated outright.
Package Structure
Differentiators
- Multilingual and multimodal scenarios are built by native-language experts to test implicit cultural understanding.
Record Schema
cultural_contextstringThe region, culture, or community the scenario is grounded in.
expected_behaviorstringThe culturally appropriate behavior or answer, written by native reviewers.
rationalestringWhy the expected behavior is appropriate, including pitfalls and sensitivities.
scenariostringThe culturally situated scenario or prompt presented to the model.
Results
| Model | Metric | Result |
|---|---|---|
| GPT-5.2 | Pass@1 |
Methodology & Quality
Methodology
SME involvement
Domain-expert curated with up to L2 review
Stressors
Implicit cultural requirements that must be inferred rather than stated directly.
Design criteria
- Test culturally appropriate reasoning beyond trivia-style factual recall.
Build assumptions
Quality approach
Known limitations
Customization & Scalability
Custom sets available. Additional volume in progress
Get Started
Request a Sample
Share your target domains and the OTS team will follow up with sample data and next steps.