Research Scientist (Remote/US/LATAM)
Anyone AI, Fully Remote +10
The role
Research Scientist, LLM Evaluations & Benchmarking
Anyone AI Labs Reports to: CEO · Remote / LatAm / US
The role Evaluation is one of the hardest open problems in AI: we still don't have reliable ways to measure what frontier models can and can't do, and the field mostly runs on benchmarks that are saturated, contaminated, or measuring the wrong thing. You'll own that problem at Anyone AI, measuring frontier model capability.
This is a research role at heart: you decide what a good evaluation is , design the benchmarks that prove it, and defend the methodology under lab scrutiny. You'll build frontier-grade evaluation packages across reasoning, coding, agents, tool use, and multi-modal — grounded in expert-verified truth, validated against multiple models, and QC'd to survive buyer-side review.
What you will do
- Evaluation research. Turn eval targets into original benchmark designs. Own the hard measurement questions: construct validity, item discrimination, headroom, reliability, contamination, and capability elicitation. Push toward evals that stay informative as models improve.