Perception Engineer

About the role

ABOUT LIGHTSPEED

LightSpeed Build Technologies is revolutionizing the construction industry through AI-powered robotics. Our flagship systems—BRUTE for automated wall panel manufacturing and DEX for on-site collaborative construction—are addressing the global housing crisis by delivering unprecedented speed, precision, and affordability in homebuilding.

POSITION OVERVIEW

As a Perception Engineer, you will own how LightSpeed's robots see. You will design, calibrate, and deploy the multi-camera systems on our robot cells and end-of-arm tooling, and build the perception that guides manipulation in real time. Your camera streams also feed our teleoperation and robot learning pipelines, so the quality of what you build directly shapes how well our robots learn. This role sits at the intersection of sensor hardware, 3D vision, and deep learning, all deployed on physical robots where reliability matters.

WHAT YOU'LL WORK ON

CAMERA & SENSOR SYSTEMS

- Select, specify, and deploy multi-camera systems (RGB, stereo, depth) for robot cells, including wrist- and tool-mounted cameras on end-of-arm tooling and dexterous hands

- Own intrinsic, extrinsic, and hand-eye calibration, and keep calibration reliable across tool changes and daily operation

- Synchronize cameras with robot state and teleoperation streams

- Work with mechanical engineers on camera placement for coverage, occlusion, and lighting

PERCEPTION DEVELOPMENT

- Estimate and track robot and end-effector pose using vision systems to verify kinematics and identify drift

- Compare 3D scans against CAD or BIM models to check part placement and as-built accuracy

- Localize robots and tools in the workspace using fiducials (AprilTag or ChArUco) and multi-view fusion

- Build object detection, segmentation, and 6-DoF pose estimation for parts, fasteners, and tools in cluttered, confined, and poorly lit workspaces

- Build point cloud pipelines (filtering, segmentation, registration such as ICP, and meshing) for manipulation and scene understanding

- Track hands and objects in egocentric video of human demonstrations

- Build CUDA-optimized perception pipelines: GPU preprocessing, custom CUDA kernels where needed, TensorRT inference, and minimal host–device copies

- Profile and optimize end-to-end latency, from photon to action, using tools like Nsight Systems

DATA & ROBOT LEARNING SUPPORT

- Deliver calibrated, time-synchronized camera data in the formats our teleoperation recording and training pipelines consume

- Partner with AI/ML engineers to define observation spaces for imitation learning and vision-language-action policies

- Set perception accuracy targets tied to task success, and build benchmarks to measure them

INTEGRATION & DEPLOYMENT

- Integrate perception with ROS2-based robot control