Nx Agents: The Next Leap in Distributed Task Execution

Nx Agents represent the next evolution of Nx Cloud's Distributed Task Execution (DTE), bringing a new level of efficiency and simplicity to your CI/CD pipelines. It takes away the complexity of configuring agents and comes with features such as scaling of agents based on the PR, flaky task re-running, and intelligent task splitting and distribution. Keep reading to learn more.

Currently in private beta, Nx Agents are slated for public release in February 2024. Don't miss the opportunity to be among the first to experience this groundbreaking tool. Sign up now for early access.

What's the Difference to DTE?

Nx Cloud's Distributed Task Execution (DTE) introduced an easy way to intelligently distribute tasks across machines, allowing for a more fine-grained distribution taking historical data as well as the task dependencies into account.

Using DTE you have to configure and instantiate your agents, which might be more or less complex depending on your CI provider. We have some guides on how to do that here.

Nx Agents take away that complexity by delegating the agent management to Nx Cloud. You can think of them as a managed version of DTE.

Keep reading to learn what the configuration and setup looks like.

Managed Agents, Seamless Configuration

Enabling task distribution with Nx Agents can be done in a single line. Simply add the --distribute-on property to the start-ci-run line in your CI pipeline configuration:

This instructs Nx Cloud to distribute tasks across 8 agents of type linux-medium-js. linux-medium-js is the name of the launch template that will be used to provision the agent. The default launch templates can be found here (there will be more once Nx Agents is publicly available).

You can also define your own "launch templates" (here's an example from the Nx repo):

Here are the available resource classes.

Intelligent Dynamic Scaling

Instead of defining

...which always runs tasks on the same amount of machines, you can also have Nx Cloud scale the number of agents based on the size of your PR.

You can then reference it in your CI pipeline configuration:

Automatic Task Splitting

Imagine you're working on an end-to-end (e2e) project using tools like Cypress or Playwright. Traditionally, to make the most of features like affected, caching, and distribution, you'd need to divide your project into smaller parts. But this approach can often be cumbersome and less efficient for developers.

Nx is on the verge of introducing a game-changing feature that enables dynamic target definitions for projects (more details to come). Paired with Nx Agents, this innovation allows you to distribute e2e tests at the file level across various agents.

This significantly cuts down the time required to run e2e tests. For instance, in the video shown at the beginning of the page, e2e test durations plummeted from 90 minutes to just 10 minutes.

Flaky Task Re-Running: Enhancing Reliability

Flaky tasks are a common headache, particularly with tests and end-to-end (e2e) tests. Nx Agents offer a solid solution to detect and automatically retry such unreliable tasks.

Nx Cloud keeps track of the targets being executed. A task, like myapp:e2e, is labeled as flaky if it shows different outcomes (status codes) for the same cache hash key. This key is an ideal task identifier, encompassing the command, environment variables, source files, and more.

Consider this scenario:

• An Nx agent runs myapp:e2e.
• Nx calculates the cache hash key for this task.
• myapp:e2e fails; Nx Cloud notes this failure along with the cache key.
• In a subsequent run, myapp:e2e is executed again.
• Nx recalculates the cache hash key, which matches the previous run's key (no existing cache since the initial task failed).
• This time, myapp:e2e succeeds.
• Nx Cloud identifies the task as flaky and stores this data temporarily.

As a result, if Nx Cloud has marked a task as flaky, it will be automatically retried, ideally on a different Nx Agent to prevent issues from any residues of earlier runs.

Sign up now for early access and be one of the first to try Nx Agents.