Senior/Staff AI Engineer, Quality & Evals (m/f/x)
Cortea AI, Berlin
The role
We are looking for an engineer with strong backend, data, and AI systems experience to build the evaluation and observability foundation for production-grade LLM agents used in complex audit workflows.
This role sits at the intersection of backend engineering, data infrastructure, and AI quality. You will build the evaluation systems that power our multimodal retrieval agents and continuously improve critical quality metrics across current and future pipelines.
You’ll work at the edge of applied AI and information retrieval, building multimodal agentic pipelines and solving hard context and agent-harness engineering problems.
This is not a traditional analytics, BI, or dashboarding role. You should expect to write production code, design data architecture, work inside backend systems, and directly improve the quality, cost, reliability, and performance of LLM-based agents.
What you will do
You will help build and operate the technical systems around our AI agents, with a focus on data infrastructure, evaluation, observability, and optimization. You will: