Member of Technical Staff - Mechanistic Interpretability

LLMs are fantastically powerful and there is a rapidly growing corpus of work devoted to understanding their internal representations and computations. We use the tools of mechanistic interpretability to enhance reinforcement learning by generating intrinsic rewards as a supplement or alternative to downstream human-generated verifiers.

• Develop methods for using mechanistic interpretability to extract useful training signals from the internal states of language models.
• Turn representations, features, circuits, and causal model behaviors into intrinsic rewards for reinforcement learning.
• Compare interpretability-derived rewards against human feedback, learned reward models, verifiers, and task-level outcome rewards.
• Design metrics and baselines for reward quality, including alignment with intended behavior, generalization across tasks, robustness, and resistance to reward hacking.