A collection of machine-learning implementation tasks designed to evaluate agents’ ability to translate learning objectives into working Python solutions. Tasks emphasize reasoning about imperfect supervision, implementing statistical methods, and producing reliable predictions under varied evaluation conditions. Automated scoring measures both predictive performance and probability quality.
Available on Request · Sample Available
MLS Bench
Execution-graded ML coding tasks that test robust learning from noisy and imperfect supervision.
Volume
0
Volume
Scalable Volume
Coverage
Agentic Workflows
0.0
%
Opus 5
Targets
MLS-Bench
Modality & Delivery
textcode→codestructured Data
Formats
Repository
What You Receive
Coverage
Weak supervision, label aggregation, context-aware learning, correlated supervision sources, class-dependent reliability, abstention, class imbalance, and probability calibration.
Package Structure
- instructions.md
- tasks.json
- environment/
- tests/
Differentiators
Record Schema
Results
| Model | Metric | Result |
|---|---|---|
| Opus 5 | Pass@1 |
Methodology & Quality
Methodology
SME involvement
Stressors
Design criteria
Build assumptions
Quality approach
- Fixed evaluation assets
- Integrity checks
- Explicit output validation
- Programmatic scoring against reference labels