Member of Technical Staff — Training Infrastructure

Causal Labs, San Francisco +1

What you will do

Design, implement, and optimize distributed training systems that scale across thousands of GPUs
Research and test parallelization strategies and numerical precision trade-offs across model scales, including for architectures that don't map cleanly onto existing LLM training stacks
Analyze, profile, and debug low-level GPU operations to maximize throughput and hardware utilization
Build reusable frameworks for checkpointing, fault tolerance, and reproducibility that stay robust under rapid research iteration
Collaborate with researchers to bring novel model architectures from prototype to full scale
Stay up-to-date on research to bring new ideas to work

We value a relentless approach to problem-solving, rapid execution, and the ability to quickly learn in unfamiliar domains.

Demonstrated proficiency with distributed training frameworks and techniques (e.g. FSDP, DeepSpeed, Megatron, Pytorch, JAX/XLA) to train large foundation models

Causal Labs, San Francisco +1

Design, implement, and optimize distributed training systems that scale across thousands of GPUs
Research and test parallelization strategies and numerical precision trade-offs across model scales, including for architectures that don't map cleanly onto existing LLM training stacks
Analyze, profile, and debug low-level GPU operations to maximize throughput and hardware utilization
Build reusable frameworks for checkpointing, fault tolerance, and reproducibility that stay robust under rapid research iteration
Collaborate with researchers to bring novel model architectures from prototype to full scale
Stay up-to-date on research to bring new ideas to work

We value a relentless approach to problem-solving, rapid execution, and the ability to quickly learn in unfamiliar domains.

Demonstrated proficiency with distributed training frameworks and techniques (e.g. FSDP, DeepSpeed, Megatron, Pytorch, JAX/XLA) to train large foundation models