Inference Intern

• Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
• Assist in building, enhancing, and scaling our runtime, including multi-node inference, intra-node execution, state management, and robust error handling.
• Contribute to optimizing routing and communication layers using our collectives.
• Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.
• Develop and leverage a deep understanding of our architecture to co-design both HW instructions and model architecture operations to maximize model performance
• Implement high-performance software components for the Model Toolkit

You may be a good fit if you have

• Progress towards a Bachelor’s, Master’s, or PhD degree in computer science, computer engineering, applied mathematics, or a related field
• Proficiency in Python, C++
• Understanding of performance-sensitive or complex distributed software systems, e.g. Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).