Engineering Manager, Private Cloud Compute Inference Routing

About the job

At Apple, iCloud Platform Engineering builds the server stack behind Push Notifications, iMessage, FaceTime, iCloud, iCloud Private Relay and Private Cloud Compute (PCC), the backbone of Siri AI and Apple Intelligence. We are looking for an Engineering Manager to lead PCC inference routing. These systems send every inference request to the right compute across a global, multi-region fleet, while keeping latency low, availability high and PCC's privacy guarantees intact. This is a technically grounded leadership role for someone who thrives at scale, can bring clarity to complex and fast-changing environments, and is energized by building both systems and teams. You'll guide architectural direction, scale operational practices and develop the engineers and leaders who will shape how Apple delivers cloud inference.

Description

Routing LLM inference at Apple's scale takes systems that adapt to changing demand, available capacity and failures while keeping latency low and accelerator utilization high. As the fleet grows across regions and cells, routing must account for capacity and health across the whole fleet, not just a single site. PCC adds a basic architectural requirement: request content stays encrypted outside trusted compute nodes. You'll lead the design of routing and scheduling strategies within this privacy model. Together with inference and security teams, you'll evaluate techniques such as cache-aware request placement and prefill/decode disaggregation. Your scope includes the inference AI gateway layer, where requests are authenticated, metered, prioritized and streamed back to clients, all without exposing request content. This work has broad reach. Every improvement to routing makes Siri AI and Apple Intelligence faster and more reliable, and makes better use of Apple's inference capacity. In your first year, you'll be expected to grow a high-performing organization, set a clear technical roadmap, and measurably improve routing latency, availability and fleet utilization.