Senior Machine Learning Engineer

About Buildkite

At Buildkite, our mission is to unblock every developer on the planet. We’ve rethought how software delivery should work and have built a platform that is fast, reliable, secure, and able to scale to the needs of the most demanding high-growth tech companies globally including Airbnb, Shopify, Canva, PagerDuty, Lyft, and Pinterest.

Job Overview

As the first Machine Learning Engineer on the Buildkite Test Engine team at Buildkite, you’ll own the end-to-end ML lifecycle — from partnering with customers to design meaningful products, to designing, deploying, and monitoring production-grade models at scale. Your work will directly reduce developer friction and improve confidence in software delivery for high-growth, engineering-led organisations. You’ll build robust inference and retraining pipelines, and ensure models are observable, cost-efficient, and impactful. This is a high-autonomy position where you’ll shape how machine learning differentiates Test Engine and meaningfully improves the developer experience.

About the Team

Test Engine helps to make test suites cheaper, faster and more reliable, reducing friction in developers' day to day work and making their working lives happier and more productive. We do this by identifying and quarantining flaky tests, providing tools to parallelise your suites to minimise build time, as well as showing trends over time of your suite and providing the ability to deep dive into what happens when an individual test executes.

What You’ll Do

Talk directly to Buildkite customers to deeply understand their problems and work together on coming up with AI/ML product solutions

Design, build and evaluate models, and different strategies to apply and use them

Help drive architectural decisions around the data stack, with a focus on scalability, cost efficiency, and maintainability

Instrument production models with observability metrics: latency, throughput, prediction drift, and data quality at inference time

Shape product direction, help define where machine learning meaningfully differentiates our product — and where simpler approaches suffice.

What You Bring

5+ years of experience in working with machine learning models, particularly feature engineering, hyperparameter tuning, model serving, monitoring, and automated retraining pipelines

Familiarity with the developer tooling space, contemporary testing practices and software engineering best practice

Strong communication skills, with a focus on telling data driven stories and bringing a data shaped lens to engineering product design

Ability to work autonomously in a fast-paced, low-process environment where you’ll need to navigate ambiguity and drive impactful decisions

Key Technical Stacks:

ML infrastructure and libraries (XGBoost for example)

Deep proficiency in SQL and Python

Strong experience with distributed data processing frameworks such as Spark, PySpark, or Flink

Experience with streaming data infrastructure (Kafka, Kinesis, pub/sub, event hubs, or similar)

Hands-on experience building and operating ML infrastructure — model registries, feature stores, serving layers, or similar

Comfort working in cloud environments (AWS) and with containerised workloads (Docker, Kubernetes) and AWS Sagemaker

✨ Why Join Buildkite

At Buildkite, we value kindness, autonomy, and collaboration. You’ll be part of a remote-first company where your work can make a meaningful impact – empowering engineers worldwide to build and deliver better software faster.

Competitive compensation and benefits package

Flexible, remote-first culture

Opportunities for professional growth, leadership, and technical ownership

Work alongside talented, passionate engineers building world-class developer tools

A collaborative, inclusive, and innovative culture where your ideas make a real impact

Equal Opportunity Employer

At Buildkite, we value diversity and celebrate all types of skills, backgrounds, and experiences. We’re dedicated to fostering an inclusive environment and providing reasonable accommodations throughout our recruitment process.

If you need any accommodations or support during the application or interview process, please reach out to us at accommodations@buildkite.com.