Research Engineer / Scientist - Storage for LLM

We are seeking a systems researcher or engineer with deep expertise in large-scale distributed storage and caching infrastructure to design and maintain a high-performance KV cache layer for large language model (LLM) inference. This role focuses on improving latency, throughput, and cost-efficiency in transformer-based model serving by optimizing the reuse of attention key-value states and prompt embeddings. You’ll work on cutting-edge AI systems problems with real-world impact, alongside a world-class team.

The role offers opportunities to publish, contribute to open-source, attend top conferences, and enjoy competitive compensation, generous research resources, and an innovation-driven culture.

• Design and implement a distributed KV cache system to store and retrieve intermediate states (e.g., attention keys/values) for transformer-based LLMs across GPUs or nodes.
• Optimize low-latency access and eviction policies for caching long-context LLM inputs, token streams, and reused embeddings.