Senior Software Engineer, GPU Code Generation for Machine Learning Inference

About the role

NVIDIA is seeking talented engineers to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. Our team develops software that accelerates state-of-the-art generative AI models for local and enterprise deployments. We solve challenging problems in GPU code generation, just-in-time compilation, and inference optimization to deliver high performance across diverse models and hardware configurations.

As a Senior Software Engineer, you will develop advanced GPU inference software for TensorRT. We’re looking for engineers with strong experience in GPU optimization and compiler development who can translate a deep understanding of GPU execution into efficient code generation and measurable improvements in generative AI inference.

What you’ll be doing:

- Design and implement efficient GPU code generation techniques in TensorRT that balance generated code performance, compilation overhead, and startup latency.

- Develop compiler optimizations, including operator fusion, dynamic scheduling, and specialization for model shapes, data types, and GPU architectures.

- Profile workloads and analyze generated GPU code to identify bottlenecks and guide compiler and runtime improvements.

- Develop robust, maintainable software in C++, and Python for tooling, testing, and performance analysis.

- Collaborate with GPU architects, kernel developers, compiler engineers, and deep learning experts to guide hardware and software design.

What we need to see:

- BS, MS, or PhD in Computer Science, Computer Engineering, or a related field, or equivalent experience.

- 4+ years of relevant software development experience, including contributions to a substantial codebase or project.

- Strong proficiency in C++ and experience building performance-critical software.

- Understanding of GPU architecture and parallel programming concepts, including memory hierarchies, thread execution, synchronization, and resource utilization.

- A rigorous approach to performance analysis and debugging, with the ability to validate numerical correctness and explain performance tradeoffs.

- Strong problem-solving and communication skills, and the ability to work independently and collaborate across teams.

Ways to stand out from the crowd:

- Experience developing or optimizing GPU kernels using CUDA, Triton, CuTe DSL, CUTLASS, or other GPU programming languages.

- Experience with MLIR compiler infrastructure, familiarity with intermediate representations, lowering, optimization passes, and GPU code generation.

- Familiarity with NVIDIA Tensor Core architecture, mixed-precision computation, quantization, or low-level GPU performance analysis using tools such as NVIDIA Nsight.

- Experience developing inference frameworks such as PyTorch, JAX, ONNX Runtime, TensorRT, TensorRT-LLM, vLLM, and SGLang.

- A track record of measurable performance improvements in production software or contributions to GPU programming and compiler projects.