Multimodal ML Engineer

About the role

TL;DR: Multimodal ML Engineer to train and ship vision, audio, video, and speech models for an AI safety platform handling 100M+ API calls/month.

About us

White Circle http://whitecircle.com is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies – simple natural-language rules that define what an AI model should and shouldn’t do. We automatically test, enforce, and continuously improve these policies at scale.

- We’ve recently raised our Series A funding round, taking our total funding to $70M. Our investors include top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others

- We process over 100M+ API calls every month

- We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model

We’re a small, highly focused team. If you want to work deeply on hard problems, see your work ship to production quickly, and influence how AI safety is actually built – you’re the one we need.

You will

- Train and fine-tune large-scale multimodal models (vision-language, audio, speech) from scratch and from pretrained checkpoints

- Extend models across modalities: image understanding, video temporal modeling, long-context processing, and streaming audio

- Design and run experiments: architecture changes, data mixes, training recipes

- Build and maintain multimodal data pipelines — from raw images, video, and audio recordings to training-ready datasets, including synthetic data generation

- Train and optimize MoE architectures for efficient multimodal inference

- Build alignment pipelines: SFT, DPO, GRPO, reward modeling — across modalities, not just text

- Optimize models for production: quantization, distillation, batching, streaming and low-latency serving

- Deploy models end-to-end: from research checkpoint to production serving

- Define evaluation metrics and benchmarks that actually matter for the product: visual QA, spatial reasoning, video comprehension, speech and audio understanding

You’ll fit right in if you

- 3+ years training large-scale deep learning models in multimodal domains (vision-language, audio, speech, or acoustic)

- Strong PyTorch skills with hands-on distributed training experience (DeepSpeed, FSDP, or similar)

- Deep experience with multimodal architectures — you understand how vision/audio encoders, projectors, and LLMs fit together (LLaVA, Qwen-VL, InternVL, Audio Flamingo, Omni Qwen, Audio Qwen, Whisper, HuBERT, Conformer, or similar)

- Hands-on with RLHF/alignment for multimodal: GRPO, DPO, reward modelling — not just for text

- Experience with video and/or audio sequence modelling: temporal modelling, long-context processing, efficient attention, streaming inference