Production Systems Engineer, Fleet AI Systems

About the job

Meta is seeking a Production Systems Engineer to lead the reliability, scalability, and operational excellence of our AI infrastructure fleet. This role sits at the intersection of large-scale AI compute systems and production engineering, supporting the hardware and software stack that powers Meta's AI training and inference workloads across tens of thousands of accelerators and servers. You will drive the strategy for fleet health, automated diagnostics, and lifecycle management of AI-optimized hardware, partnering closely with hardware design, silicon, software, and data center operations teams to ensure our AI systems are production-ready and operating at peak efficiency.

Responsibilities

Define and drive the long-term technical strategy for fleet health monitoring, automated remediation, and lifecycle management of AI compute infrastructure at hyperscale Lead cross-functional initiatives spanning hardware design, firmware, systems software, and data center operations to bring new AI server platforms from pre-production validation through mass deployment Architect and oversee development of large-scale test automation frameworks for AI accelerator systems, covering GPU, TPU, and custom silicon configurations across diverse fleet topologies Establish organizational standards for root cause analysis methodologies, failure mode classification, and escalation workflows for AI system incidents at fleet scale Drive vendor and partner engagement on hardware specifications, firmware quality, and test methodology improvements to continuously raise the reliability bar for AI infrastructure Develop and champion data-driven visibility platforms that surface fleet health signals, predictive failure indicators, and capacity risk across AI training and inference clusters Lead ambiguous, high-impact investigations into systemic hardware and software failures affecting AI workload availability, producing durable solutions and documented learnings Mentor and provide technical direction to other engineers across the production systems organization, shaping engineering practices and raising the collective capability of the team Collaborate with capacity engineering and AI platform teams to translate fleet reliability requirements into hardware procurement, qualification, and deployment criteria Influence the roadmap of internal tooling and automation systems to reduce mean time to detection and resolution for AI fleet incidents at scale