Robot Learning Engineer
The role
You'll own the full loop from synthetic data to deployed policy: the Isaac Lab/Mimic/Cosmos pipelines that generate and augment training data, and the models — VLA finetunes, custom vision encoders, diffusion/flow-matching action heads — that train on it.
What you'll do
- Design and maintain SDG pipelines in Isaac Lab and Isaac Mimic — scene and asset setup, demonstration generation and multiplication, domain randomization, sensor/observation configuration.
- Integrate Cosmos (or similar world-foundation-model tooling) to improve visual realism of synthetic data and reduce the sim-to-real gap.
- Build and tune co-training pipelines that mix real demonstration data with synthetic data — owning the data ratios, curation, and evaluation that determine whether the mix actually helps.
- Finetune existing VLA models (e.g., π0) on our tasks and embodiments.
- Design and train custom, task-specific vision encoders and action heads (diffusion policy, flow-matching policy) for visuomotor tasks.
- Run sim-to-real validation — take a policy that works in Isaac Lab and diagnose why it doesn't transfer, iterating on both the model and the data generation side as needed.
- Build and maintain the supporting training infrastructure: data loaders, logging, evaluation harnesses, ablation tooling — whatever lets you iterate fast.
What we need
- 3–4 years of hands-on experience training deep learning models for vision or robotics. You've trained, debugged, and shipped real models — not just run tutorials or notebooks.
- Strong PyTorch fundamentals: you can implement a custom architecture, debug an unstable training run, and reason clearly about loss functions, objectives, and optimization choices.
- Direct experience with imitation learning / behavior cloning, ideally including diffusion or flow-matching policies specifically (action chunking, denoising or velocity objectives, multimodal action distributions).
- Production-adjacent Python engineering skills — code that's testable and debuggable, not research-script quality.
- Some prior exposure to a robotics simulator for RL or imitation learning — Isaac Sim/Lab, Isaac Gym, MuJoCo, PyBullet, or similar. We care more about evidence you can pick up a new simulator fast than about which one specifically.
- Genuine interest in owning the data side, not just the model side. This role doesn't work if you want to treat the data pipeline as someone else's problem.
Nice to have
- Hands-on Isaac Lab / Isaac Mimic / Omniverse experience specifically.
- Experience with NVIDIA Cosmos or other world-foundation-model / video-generation tooling for synthetic data augmentation.
- Experience finetuning a published VLA (π0, OpenVLA, or similar).
- Familiarity with USD scene composition, domain randomization frameworks, or debugging GPU-parallelized physics.
- Published work or open-source contributions in robot learning (CoRL, RSS, ICRA, NeurIPS, or LeRobot/Isaac Lab-adjacent repositories).
How we'll evaluate you
- A project deep dive: walk us through one thing you trained end-to-end, including what broke and what you changed.
- A short, scoped take-home in our actual stack — not LeetCode. Likely something like finetuning a small policy on a provided dataset, or diagnosing a sim-to-real-style discrepancy.
- A technical conversation with the engineer you'd work most closely with, going deep on one or two specific decisions from your past work.
