AI Engineer, Post-Training & Evaluation

About the role

The role

Own the models behind Coval’s judges. Coval is the evaluation platform for conversational AI: we help enterprise teams simulate, monitor and improve voice and chat agents before those agents ever reach a customer. Because our product is the judge, this role sits at the center of the company.

You’ll own the model side end to end. You’ll design and ship the LLM-as-a-judge evaluations that score voice and chat conversations at scale, then sharpen them with LoRA fine-tunes on open-weight models and non-tuning work like tool use and context engineering. Your improvements ship directly into the product enterprises rely on to launch dependable voice agents.

What you’ll do

- Design and ship the LLM judges that score conversations: rubrics, judge prompting, and calibration against human raters

- Build the reliability tracking that keeps judges honest from release to release

- Run post-training where training beats prompting: LoRA adapters on open-weight models, trained on data distilled from our own scored conversations

- Everywhere else, do the non-tuning capability work: tool use, structured outputs, and context engineering

- Keep asking which lever actually moves the metric, and focus there

What we’re looking for

- 4+ years as an engineer, with strong systems instincts

- A track record of shipping measurable quality gains by pairing rigorous measurement with targeted training, not vibes and prompt tweaks

- You’ve built eval infrastructure you actually trust

- You’ve run LoRA/SFT experiments end to end, including preparing your own data rather than relying on clean data from other teams, so you know which lever really wins

- High agency: on a small Series A team, nobody hands you tickets. You figure out what matters next and drive it there

- Ambitious and kind. You’ll be part of the group that founds our engineering org, sets its bar and grows the team

See the posting on Coval's site