Push a one-line fix. Then watch CI grind through forty minutes of tests, ninety-five percent of which never had a chance of touching what you changed. You already know the handful that mattered. The test suite doesn't — so it runs everything, every time, just in case.
That "just in case" is the most expensive habit in software delivery. Every engineering team pays it, because the alternative — knowing which tests actually matter for a given change — has been too hard to get right.
We're building the team that gets it right. This role sits at the centre of it.
Test Engine already ingests billions of test runs. We can see the tests, the code underneath them, and how the two move together — at a scale very few people ever get to work with. The raw material for the answer is already here. Nobody's turned it into predictions yet.
That's the step to take: for a given change, work out the slice of tests most likely to fail, and run only those. Get it right and teams stop re-running what hasn't changed, and spend that time where it counts — like fixing the two percent of tests most likely to break.
It's a genuinely difficult ML problem — sparse signal, cold-start on new repos, generalising across languages and frameworks, and latency tight enough to sit in the critical path. It's also close to a blank page. There's no ML org above you setting the direction — you'd set it. And not alone: we've just hired another ML engineer, so there's someone to think out loud with from day one.
Machine learning in Test Engine, end-to-end — the strategy, the architecture, and the models running in production.
That means shaping the whole path: pulling features out of code changes and test history, training and evaluating models, building the serving layer that keeps predictions fast, and closing the loop so the system keeps improving. You'd make the trade-offs that matter — accuracy versus latency, what happens when confidence is low — and build the platform underneath so the next model into production is quick and repeatable, not a one-off.
You've taken ML models the whole way — from rough idea to something running reliably in production, monitored and retrained, owned rather than handed off.
Two things matter more than any specific tool:
- You've built ML that generalised. Not one clever model — a repeatable approach that worked across more than one use case.
- You're comfortable where the signal is noisy. Classification, ranking, prediction — problems where the data doesn't hand you the answer.
Day to day you'll live in Python and SQL, on AWS, with containerised workloads and data-at-scale tooling (Spark, Flink, or similar). Experience with code analysis, CI/CD systems, or ranking problems is a real head start — a bonus, not a bar.
The one thing we won't budge on: you've shipped and owned ML in production. Prototyped and handed off doesn't count here.
You're likely a strong fit if you:
- Get energised by a blank page and want to be the one who fills it
- Care more about models working in production than papers about models
- Do your best work async, with deep focus and real autonomy
This probably isn't the right role if you:
- Want an established ML org around you for direction and review
- Prefer research and experimentation over shipping and operating
- Need close scaffolding — flat and high-autonomy means less of it
We'd rather you know that now than three interviews in.
- Frontier work. CI/CD is becoming the next bottleneck in the AI era, and Buildkite is built for that moment.
- Real scale. The world's leading engineering teams ship software to over a billion daily users through Buildkite. Your models sit in their critical path.
- Ownership. ~150 people, flat structure, and you're the most senior ML person here — influence you don't get where the ML org is three layers deep.
- Remote, properly. We've worked this way since 2013 — async, built for deep focus, with genuine overlap across ANZ and US-Pacific.
Every application gets a response. If this is the problem you've been wanting to get your hands on, apply now, or reach out with questions first.
Our Engineering teams are based in the ANZ/PST region. This is a conscious decision, as it allows us to move quickly and minimise fully async work. So whilst Buildkite is a fully remote company, this doesn't mean that we hire in every location. Please be aware that if you're applying from outside of this region, we unfortunately aren't in a position to hire you. Currently Buildkite is not in a position to offer sponsorship.
