Acceleration or AWS CODEBUILD: Parallel Test Speeting is now available | Amazon Web Services

I am excited to announce that the AWS Codebuild is now supporting parallel tests, so you can run your test kits competitively and shorten the Nordically assembly times.

With the demo project, which I wrote for this post, the total test time dropped from 35 minutes to 6 minutes, included the time to secure the surroundings. These two screenshots from the AWS management show show the difference.

Sequential design of the test kit

Parallel design of the test set

Very long test times represent a significant challenge in conducting continuous integration (CI) on a scale. As the projects grow in the complexity and size of the team, the time required to perform non -inflammatory test apartment can increase dramatically, leading to prolonged pipes. This not only delays the delivery of new features and corrections of errors, but also prevents the developer’s productivity by forcing them to wait for the results of the assembly before processing with their tasks. I have experienced a pipe that took up to 60 minutes to run just to fail in the last step, required a total re -re -re -re -delay. These lengthy cycles can disrupt the trust of developers in the CO process, contribute to frustration and finally slow the entire software delivery cycle. In addition, long -term tests can lead to resources reduction, increasing Belcaus costs of unnecessary computing force and reducing the overall efficiency of the development process.

With a parallel test in Codebuild, you can now have your tests in several assembly calculations. This feature implements access to Sharard, where each assembly node independently performs a subset of the test kit. Codebuild Providids Environment variable that identifies the current node number and the total number of nodes used to determine which tests should run. There is no node or coordination of control assembly between nodes at the time of assembly – each node works independently of the assigned part of your tests.

You want to enable the division test, configure the dose fanut section in your buildspec.xmlSpecification of the required level of parallelism and other partners. In addition, use the End Test tool in the step, along with the testing commands and the selected distribution method.

The tests are divided on the basis of a specific shading strategy. codebuild-tests-run Offers two Shararding strategies:

• The same distribution. This strategy sorts the test files alphabetically and distributes them into pieces equally across parallel test exsens. Changes in names or a number of test files can assign files across SHARDS.
• Stability. This strategy corrects distribution via shards using ARES algorithm. Maintains an existing assignment of files-shad when new files are added or deleted.

In parallel performing Codebuild tests automatic merger of test messages. When combining the Codebuild Consolidates automatic test message, it tests messages into a single test summary and simplifies the analysis of the results. The merged message includes aggregated passenger/failure states, test duration and failure details, reducing the need for manual messages. You can see the merged results in the Codebuild console, load them with the AWS (AWS Cli) command line, or integrate them with other testing tools to streamline the test analysis.

Let’s see how it works
Let me show how to implement parallel testing in the project. For this demo I created a very basic project Python with hundreds of tests. To speed up, I asked Amazon Q Developer on the command line to create a project and 1,800 test cases. Each test case is in a separate file and completes one second. The launch of all tests in the sequence requires 30 minutes, with the exception of the environmental time.

In this demo I will launch a test set on the ten computing surroundings in parallel and measure how long it takes to start the set.

For this purpose I added a buildspec.yml File to my project.

version: 0.2

batch:
  fast-fail: false
  build-fanout:
    parallelism: 10 # ten runtime environments 
    ignore-failure: false

phases:
  install:
    commands:
      - echo 'Installing Python dependencies'
      - dnf install -y python3 python3-pip
      - pip3 install --upgrade pip
      - pip3 install pytest
  build:
    commands:
      - echo 'Running Python Tests'
      - |
         codebuild-tests-run \
          --test-command 'python -m pytest --junitxml=report/test_report.xml' \
          --files-search "codebuild-glob-search 'tests/test_*.py'" \
          --sharding-strategy 'equal-distribution'
  post_build:
    commands:
      - echo "Test execution completed"

reports:
  pytest_reports:
    files:
      - "*.xml"
    base-directory: "report"
    file-format: JUNITXML 

Three parts can be highlighted in the Yaml file.

First is here build-fanout According to the section batch. Tea parallelism LOCD Say Codebuild how much test environment is in parallel. Tea ignore-failure The command indicates that failure in some of the tasks of Fanout can be ignored.

Secondly, I use pre -installed codebuild-tests-run Command to run tests.

This command receives a full list of test files and decides which tests must be launched in the current node.

• Use sharding-strategy Choose an argument between an equally distributed or stable distribution, as I explained before.
• Use files-search Argument to hand over all the files that are candidates for running. We recommend using the provided codebuild-glob-search The performance command, but any tool to search for files such as FIND (1), will work.
• I’ll go through the news test command and start a Shard with test-command argument.

And finally reports The Codebuild Instruction section collects and merges the test messages on each node.

Then I open the Codebuild console to create a project and configuration of the building dose building for this project. There is nothing new, so I save details. The documentation has all the details that can begin. Parallel testing works on batch reports. Remember to configure your project to start in a dose.

Now I am ready to start the test kit. I can commit a new code on my Github restitization or run the assembly in the console.

After a few minutes I see a message about the status of different assets; With the condition for each Gold Shar test environment.

After completing the test I select Postponement Card to access the merged test messages.

Tea Postponement It aggregates all test data from all shards and maintains history for all assemblies. I choose my last assembly in The history of the report Part to access a detailed message.

As expected, I see aggregated and individual condition for each of my 1,800 test boxes. In this demo, everyone goes through and the news is green.

1,800 Demo Project tests take every second to complete. When I start this test gradually the test, it took 35 minutes to complete. When I start the test set in parallel to ten calculations around, it took 6 minutes to complete, including the time to secure around. The parallel run lasted 17.9 percent of the sequence time. Message numbers will vary according to your projects.

Other things to know
This new ability is compatible with all test framework. The documentation included an example for Django, Elixir, Go, Java (Maven), JavaScript (is), Basin, PHPUNIT, Pachest, Ruby (Cucumber) and Ruby (RSPEC).

For test frames you have received lists of separated spaces, codebuild-tests-run CLI provides a flexible alternative through CODEBUILD_CURRENT_SHARD_FILES Variable environment. This variable contains a list of test paths separated by a new line for the current share assembly. You can use it to adapt to different requirements for the test frame and the names of the format test files.

You can further customize how tests are divided across the environment CODEBUILD_BATCH_BUILD_IDENTIFIER Variable environment that is automatically set in each assembly. You can use this technique to implement parallelization or optimize specific to the frame.

Prices and availability
With a parallel test, you can now complete your test apartment for a fraction of the previously necessary time, speed up your development cycle and improve your team’s productivity.

The parallel test version is available on all three calculated Codebuild: On-Demand, Reserved Capacity and AWS Lambda Compute.

This ability is now available in all AWS regions where Codebuild is offered, without additional costs over the standard Codbuild prices for used computing resources.

I invite you to try parallel tests in Codebuild today. Visit the AWS Codebuild documentation to learn more and start with the parallelization of your tests.

– seb

PS: Here is the challenge I use to create a demo application and its test set: “I write a blog post to announce Codebuild Parlll testing. Write a very simple APT bag that it is tested, each test in a separate test file.

How’s the Blog of news? Take this 1 minute survey!

(This survey is hosted by an external company. AWS processes your information as described in the AWS Privacy Notice. AWS will own data collected via this survey and will not share the collection of Lissel survey.)