Senior Software Development Test Engineer - AI

About the role

ABOUT TEKION:

Positively disrupting an industry that has not seen any innovation in over 50 years, Tekion has challenged the paradigm with the first and fastest cloud-native automotive platform that includes the revolutionary Automotive Retail Cloud (ARC) for retailers, Automotive Enterprise Cloud (AEC) for manufacturers and other large automotive enterprises and Automotive Partner Cloud (APC) for technology and industry partners. Tekion connects the entire spectrum of the automotive retail ecosystem through one seamless platform. The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers/dealers and consumers. With its highly configurable integration and greater customer engagement capabilities, Tekion is enabling the best automotive retail experiences ever. Tekion employs close to 3,000 people across North America, Asia and Europe.

About the Role We are looking for highly motivated Senior AI Software Development Engineers in Test (AISDET) to join Tekion’s ML Engineering team. Our team builds and ships the machine learning  models and AI agents that power the Automotive Retail Cloud (ARC) — from Service AI Agents and F&I automation to the AI-powered Analytics Agent — trained on real-time data spanning 1,700+ dealership rooftops. In this role, you will be responsible for building confidence in the quality, correctness, and reliability of the models and AI agents we deliver. You will work closely with ML Engineers, Data Scientists, the AI Platform team, Product Management, and Applied AI teams to define robust validation strategies, automate testing across the model lifecycle, and ensure our AI features meet the highest quality bar before they reach dealers and their customers.Because you are testing AI systems, you will go beyond deterministic assertions: you will design evaluation frameworks for non-deterministic model and agent outputs, measure accuracy and consistency, catch hallucinations and regressions, and continuously expand coverage. You will also use AI and LLMs yourself — to generate test cases, synthetic data, and automation scaffolding — raising both the speed and the depth of quality engineering.

What You’ll Do

- Develop a deep understanding of Tekion’s AI agents, ML models, business domains, and the data that feeds them.

- Own end-to-end quality for the ML models, AI agents, and AI-powered features shipped by the ML Engineering team.

- Design and implement automated testing frameworks for model inference, prompt pipelines, agentic workflows, RAG/retrieval systems, and the APIs that serve them.

- Validate model correctness, accuracy, latency, and behavioral consistency across model versions, prompt changes, and upgrades.

- Define and execute evaluation strategies for AI/LLM-powered capabilities by:

- Creating and curating evaluation datasets (evals) and ground-truth sets

- Measuring response accuracy, relevance, and consistency

- Identifying hallucinations, unsafe outputs, and edge cases