Merging Playwright Multiple Sharded Reports on GitLab

Hasan Gürhan
3 min readMay 27, 2024

In my previous blog post, we discussed how to split our Playwright test suite into multiple shards to speed up the build pipeline.

That setup resulted in multiple test reports, one for each shard. The main drawback of this approach was the need for a unified view of all tests, making it more difficult to locate specific test reports. For example, if a test failed, you first needed to identify the shard to find the corresponding test report.

Now, with the introduction of the merge-reports command, we can achieve a streamlined pipeline with a single, consolidated test report. This makes it much easier to get an overview of all test results.

Updating the Test Reporter

Let’s proceed with updating the test reporter. Currently, the reporter outputs test results into an HTML file, but HTML reports cannot be merged into a single report. Instead, we need to export the test results into a format that can be later processed and merged.

To accomplish this, we’ll modify the reporter within the Playwright configuration to generate a blob file instead of an HTML file. Additionally, we’ll change the output directory containing the test result files from playwright-report to blob-report.

This modification will yield a zip file containing a JSON file with the test output.

Playwright Configuration File

import type { PlaywrightTestConfig } from "@playwright/test";

const config: PlaywrightTestConfig = {
reporter: process.env.CI ? 'blob' : 'html',
};

export default config;

Generating a Test Report from the Blob Files

Now, let’s proceed with creating an HTML test report. The final step involves merging the different blob files. Fortunately, we don’t need to do this manually; instead, we can leverage the new merge-reports the command provided by the Playwright.

Simply pass the command to the location where your blob files are stored

npx playwright merge-reports --reporter html ./blob-reports

By executing the merge-reports command, all the test results will be consolidated into a single, comprehensive HTML report. This simplifies the process of reviewing test outcomes and identifying any issues.

Merging HTML reports in GitLab CI/CD Pipeline Configuration

GitLab CI/CD includes pipeline stages responsible for running tests and merging reports. Below, we explain the GitLab CI script used to merge reports and send them to Slack.

reports:
stage: merge-reports
image: mcr.microsoft.com/playwright:v1.43.0-jammy
before_script:
- npm ci
script:
- mkdir -p all-blob-reports
- cp -r blob-report/* ./all-blob-reports/
- npx playwright merge-reports --reporter json ./all-blob-reports > merged_tests_results.json
when: always
artifacts:
paths:
- merged_tests_results.json

Conclusion

In this article, we detailed the process of merging Playwright test reports after sharding by automating this process using a GitLab CI/CD pipeline, we ensure that test results are promptly shared with team members, making the test process more efficient and manageable. We can send a message to Slack using the merged JSON file (merged_tests_results.json) or use a test report tool. I will mention about this in another article, do not forget to follow :)

--

--