Production Systems Engineer, Sustaining
Meta, Menlo Park, California +1
About the job
Meta is seeking an experienced Production Systems Engineer to join our Release to Production (RTP) team. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The RTP team is responsible for the Hardware Lifecycle of all Meta servers including pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues. RTP Engineers work closely with hardware designers, system manufacturers, component vendors, capacity engineering, production engineering, Meta services, and data center operations teams to test systems before release to our production data centers, and to track the health and life cycle of servers in production.
Responsibilities
Develop robust, scalable, reliable practices for supporting AI and HPC infrastructure at scale Interface with external vendors and internal hardware, mechanical, power, thermal, manufacturing and software engineers to understand system architecture to develop and execute the test suites for various architectures Proactively create experiments and tooling to detect and diagnose hardware/firmware/software health issues Implement sustaining workflows across hardware and software stacks, develop and communicate sustaining practices internally Troubleshoot, diagnose and root cause of system failures and isolate components and failure scenarios while working with internal and external stakeholders Drive necessary discussions with external and internal teams on test specification and methodologies to improve test quality on an ongoing basis