Software Dev Engineer II, SageMaker Compute Governance

About the role

Training and running large AI models takes enormous amounts of expensive accelerated compute, and most of it sits idle. Across the industry, a large share of GPU capacity is paid for but never does useful work because jobs cannot find it, cannot share it, or cannot see it and use it efficiently. We build the services that closes that gap.

We are looking for a Software Development Engineer to help build the systems that put idle or inefficiently used GPUs to work. You will design and build distributed backend services that let organizations pool capacity across many clusters and accounts, route it to the workloads that need it most, and see exactly how it is being used. This is deep systems work at the intersection of large-scale distributed systems, container orchestration, and cloud infrastructure, and it directly changes how much value customers get from every GPU they pay for.

You will own services end to end, from design through launch and operation in production, and your work will ship to some of the largest AI compute customers in the world.

Key job responsibilities

Design, build, and operate distributed backend services that manage and share accelerated compute across multiple clusters and accounts.

Build scheduling and capacity-sharing capabilities that move compute to the highest-priority workloads and reclaim it when priorities change.

Build zero-touch usage and guidance systems that collect, aggregate, and report how compute is consumed, at scale and in near real time.

Own your services in production: define operational metrics, respond to issues, and continuously improve reliability, latency, and cost.

Write clear design documents, participate in design and code reviews, and raise the engineering bar for the team.

Collaborate with product, design, and partner engineering teams to turn customer problems into shipped features.

Mentor other engineers and contribute to a culture of ownership, quality, and customer focus.

A day in the life

Most days start with a short team stand-up, then it is mostly building. You might design a service that lets one cluster borrow idle capacity from another, then write it up for a design review where the team pokes holes in it before any code is written. Hard problems get their own deep-dive whiteboard session. Other days lean toward operations, chasing down why a workload did not get the capacity it should have. You will not be handed fully specified tasks; you will have the independence to help decide what to build, then build it.

About the team

We help customers get the most out of the accelerated compute they pay for. Our mission is simple to say and hard to do: make sure every accelerator a customer pays for is doing useful work, and make that easy to see and control. Cost reduction is one important part of that, but the bigger idea is the engine that delivers compute to the workloads that need it most, in as little time as possible.