About us
FAR.AI http://FAR.AI is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. Our mission is to facilitate breakthrough AI safety research, advance global understanding of AI risks and solutions, and foster a coordinated global response.
Since our founding in July 2022, we've grown to 50+ staff https://www.far.ai/about/team, published 40+ academic papers https://scholar.google.com/citations?user=FVJ24k8AAAAJ, and convened leading AI safety events https://far.ai/events/. Our work is recognized globally, with publications at premier venues such as NeurIPS, ICML including a Best Paper Honorable Mention in 2026 https://icml.cc/virtual/2026/oral/71065, and ICLR, and features in the Financial Times https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1, Nature News https://www.nature.com/articles/d41586-024-02218-7, Wired Magazine https://www.wired.com/story/jailbreaking-ai-models-google-anthropic-openai-spacexai/ and MIT Technology Review https://www.technologyreview.com/2020/02/28/905615/reinforcement-learning-adversarial-attack-gaming-ai-deepmind-alphazero-selfdriving-cars/.
We conduct pre-deployment testing on behalf of frontier developers such as OpenAI, independent evaluations for governments including the EU AI Office https://www.far.ai/news/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium, and publish the AI Security Leaderboard https://leaderboard.far.ai/ based on our red-teaming expertise. We help steer and grow the AI safety field through developing https://arxiv.org/abs/2405.06624 research https://arxiv.org/abs/2506.20702 roadmaps https://www.researchgate.net/publication/396910034_Open_Technical_Problems_in_Open-Weight_AI_Model_Risk_Management with renowned researchers such as Yoshua Bengio; running FAR.Labs https://www.far.ai/programs/far-labs, an AI safety-focused co-working space in Berkeley housing 40+ members; and supporting the community through targeted grants https://www.far.ai/programs/grantmaking to technical researchers.
FAR.AI http://FAR.AI is hiring a Research Lead to develop and lead our work on pre-training safety, shaping models’ capabilities and internal representations at their source, rather than trying to fix them after the fact.
Our initial focus is capability control: removing harmful capabilities while preserving benign ones. We see this as a promising way to prevent misuse of open-weight models in areas such as CBRN and cyber by removing offensive capabilities, and reducing loss-of-control risks by removing knowledge of oversight mechanisms. We will validate approaches like pre-training data filtering at scale, drive adoption of successful methods, and explore techniques such as gradient routing and unlearning..
We are scaling methods like Deep Ignorance https://deepignorance.ai/ by over an order of magnitude (>100B parameter models with >1T tokens). You will direct this work, partner with our red team to stress-test the resulting models, and analyze how well the methods scale to frontier systems.
About far.Research
We explore promising research directions in AI safety and scale up only those showing a high potential for impact. When an approach proves effective, we develop it into a minimum viable demonstration and work with AI developers and governments to support real-world adoption.