Research Engineer, Benchmarks

About the role

ABOUT THE ROLE

Join a small, technical team at an early-stage AI evaluation company building benchmarks for frontier AI agents. You will own the design and implementation of rigorous, domain-specific evaluations that help research teams and customers understand real-world agent performance.

WHAT YOU'LL DO

- Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.

- Work with subject-matter experts to translate real workflows into realistic tasks and evaluation criteria.

- Build reliable infrastructure for running models and agents against benchmark tasks at scale.

- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.

- Validate how benchmark results relate to real-world evaluations and user needs.

- Write clear technical documentation and reports communicating benchmark methods and results.

WHAT WE'RE LOOKING FOR

- Two to four years of experience in research engineering, machine learning engineering, or a related technical role, including at least two years building or delivering AI benchmarks, evaluation infrastructure, or agent environments.

- Strong proficiency in Python, Docker, and Linux for research or production infrastructure.

- Hands-on experience designing, implementing, and operating evaluations for AI agents or large language models.

- Experience building infrastructure to run models or agents reliably, and developing metrics, statistical analyses, or validation studies.

- Ability to collaborate with subject-matter experts and analyze workflows across different domains to create useful evaluation tasks.

- Strong technical writing, attention to detail, first-principles reasoning, and comfort working independently in a fast-paced startup.

- Published research or technical writing on benchmarking, model evaluation, or failure modes is valued. Reinforcement learning pipelines, data generation, and agent evaluation experience are also relevant.

COMPENSATION & BENEFITS

Base salary: $100,000 to $170,000 USD annually. Visa sponsorship is available.

LOCATION

On-site in Singapore, Singapore.

See the posting on Clera's site