Introducing Test Engine Workflows

I’ve been looking at Buildkite lately and I’m excited to share a new feature in Buildkite Test Engine: Workflows! This capability is designed to help teams of all sizes streamline their testing process and maximize productivity, by automatically detecting flaky patterns, surfacing them in Test Engine, and triggering custom responses.

What are Test Engine workflows? Workflows help developers detect, manage, and act on flaky tests with precision and efficiency. They provide a flexible foundation for creating automated responses to the testing patterns that matter most to your organization.

Here’s what you can configure with workflows:

• Monitors: Heuristics that detect flaky behavior. Today, Test Engine supports three: transition count, passed on retry, and probabilistic flakiness.
• Actions: Automated responses that run when a monitor triggers. You can label tests, mute or skip tests, send Slack or webhook notifications, or create issues in Linear.
• Tag filters: Optional filters to narrow which executions a monitor evaluates (for example, only monitoring results from your main branch or a merge queue branch).

Why it matters? Many teams get bogged down and postpone dealing with flaky tests. The typical response is to retry or ignore these tests to keep engineering teams unblocked - but neither approach is ideal.

Some advanced teams invest in custom solutions to maintain high confidence and streamline testing. Workflows address these common flaky test pain points by providing more sophisticated detection methods, fine-grained control over how you respond to different types of flaky tests, and customizable actions that connect directly to your existing tools and processes.

Ways you can use workflows

• Catch flaky tests early: Add monitors to surface unreliable tests before they slow down your pipeline.
• Respond automatically: Configure actions to label tests, quarantine them by changing their state, or notify your team the moment flakiness is detected.
• Exercise fine-grained control: Tailor how you detect and respond to flaky tests with precision. Create separate workflows for different test types, teams, or environments.

Get started today Workflows are included in Test Engine and you can try this feature today in public preview. Customers on Pro and Enterprise plans can currently use up to three workflows per suite.

Watch the demo below, it could be useful

#CI #CD #DevOps #SoftwareTesting #TestAutomation #QualityEngineering #Buildkite #ContinuousIntegration #DeveloperProductivity #EngineeringLeadership