Senior Software Engineer - Data Infrastructure

About the role

About us:

Calder is solving the data bottleneck for embodied AI and world models. Today's AI models lack the real-world, 4D data needed to reason on spatial problems and operate in physical environments. We capture, process, and deliver high-fidelity human-generated spatial data at scale, providing the substrate that trains the next generation of generalizable models for manipulation, navigation, and interaction.

ABOUT THE ROLE

We're looking for a Senior Data Infrastructure Engineer to help build and run the petabyte-scale data platform behind Calder's spatial data products. Every capture — robot rigs, egocentric headsets, multi-camera video, depth, IMU — and everything derived from it — reconstructions, Gaussian Splats, labels, training datasets — lands in this platform, gets processed and versioned, and is served to customers and training clusters. You'll work on storage layout, table formats, versioning, ingest, and the distributed processing that turns raw sensor data into training-ready datasets, and help the platform hold up as volume grows by orders of magnitude. This is a hands-on role: you'll write, deploy, and operate core parts of the platform alongside the team. We're looking for someone who has personally built and run a PB-level platform in production, not only someone who has used one.

- Build and scale Calder's PB-scale data lake / lakehouse for multimodal spatial data — object storage layout, partitioning, open table formats (Iceberg or equivalent), catalogs, and metadata

- Work on data versioning and lineage (LakeFS or equivalent): branches, commits, reproducible dataset snapshots, and provenance from raw capture to customer delivery

- Build high-throughput, fault-tolerant ingest for field and lab captures — upload, validation, checksumming, content-addressed deduplication, and safe promotion through bronze → silver → gold

- Build and operate large-scale distributed processing across CPU and GPU fleets — video decode and frame extraction, sync and calibration application, SLAM / reconstruction / 3DGS batch jobs, and model-based auto-labeling (pose, depth, detection)

- Build and maintain data pipelines at scale: idempotent jobs, retries, backfills, partitioned reprocessing, and clean recovery when a multi-day run fails halfway

- Build the dataset curation and query layer — metadata indexing, SQL over the lake (Trino / DuckDB / Spark SQL), and search so researchers and customers can slice data without copying it

- Keep GPUs fed: sharded training formats (WebDataset, Mosaic Streaming, or similar), caching, prefetching, and efficient cross-region movement between capture sites and training regions

- Work on platform reliability, observability, and cost on AWS — S3 tiering and lifecycle, egress, spot and right-sizing, capacity planning, and alerting

- Contribute to infrastructure-as-code (Terraform) and CI/CD for the data platform; write design docs and runbooks so others can operate it without tribal knowledge