29. 12. 2023 Emil Fazzi Development, NetEye

Playwright Tests on the NetEye Guide

During the process of developing and improving the official NetEye user guide, some bugs regarding the display of the guide on mobile devices gave us the opportunity to innovate the development process of our product, extending the testing phase prior to the release of new versions of the NetEye user guide. In this blog post, we’ll take you behind the scenes of the new addition to our workflow, a new testing phase seamlessly integrated into our release pipeline.

The recent analysis and the first implementations of using Playwright for some tests mainly concerning NetEye allowed us to also develop this solution for the testing phase involving our user guide. In this way, it will be possible to add a simple playwright test that will verify that the behavior of the NetEye user guide has remained the desired one even after the addition of the new feature or following the resolution of another bug in the NetEye user guide.

The real challenge in implementing the new Playwright tests for the user guide was to build an infrastructure that enables us to easily implement the tests and run them locally. Understanding the importance of empowering developers with the tools they need for efficient debugging and manual testing, we planned the reach of this testing phase to run locally on their laptops.

This deployment approach not only accelerates the development lifecycle but also ensures that potential issues are identified and addressed well before the merging of new features into the user guide. The real challenging task, apart from designing a solution that was easily adopted in the development phase, was to build a system that can be easily and smoothly integrated in OpenShift, where the release pipelines for the NetEye user guide will be run.

Since we wanted to develop a unique solution for the two use cases (the local execution of Playwright tests and automatic execution in the OpenShift release pipelines), we decided to develop a containerized solution that will work out of the box. The container environment will execute Playwright tests on a target (the NetEye user guide in our case) and export all the interesting outcomes of the testing phase.

The solution should be as flexible as possible and easy to execute in the case of a needed local check before the official commit: for this last use case, if someone wants to debug an unexpected behavior of the user guide, we want our solution to provide all necessary information that Playwright extracts from a failed test such as the video and error log of the test case.

Going a little bit into the details of the development of the architecture, we created a personalized image starting from the official playwright Docker image (https://playwright.dev/docs/docker), preparing the environment to execute Playwright tests. Given the modularity of the containerized solution it was quite simple to add a task for executing Playwright tests in OpenShift and adding it as a step of the release pipelines. Since we created both the custom Docker image and the OpenShift task to be as general purpose as possible, they can be utilized both in the NetEye guide and NetEye product testing pipelines.

Finally, in order to develop a solution that was easy to adopt even in local environments while developing and maintaining the NetEye guide, we decided to utilize the same Docker image used by the OpenShift testing phase in the release pipeline. This choice is not only driven by the important standard of keeping the same methodology across different scenarios whenever possible, but also by the best practice of executing every test in the same way it’s executed during the official testing phase in the release pipeline. This simple rule will increase efficiency and avoid inconsistencies during a debugging scenario.

In order to facilitate developers using the customized image for executing playwright tests, we developed a simple script that performs some basic operations, such as initializing and starting the custom playwright docker image with all the required custom parameters, such as the URL of the local NetEye guide that the developer wants to test, the folder to save the test results in, and the network between the NetEye user guide container and the playwright test execution container. This way, once a fellow developer has locally built the NetEye guide to see how the newly developed features fare on the NetEye guide, a simple command will perform the required tests to ensure everything is aligned and ready to be committed.

This was our journey through the creation of a baseline architecture for future testing improvements in both the NetEye guide and the NetEye product: it simply started with some minor bugs in our NetEye guide on certain mobile devices and finished with a new feature for our continuous integration system that facilitates the development and maintenance process.

These Solutions are Engineered by Humans

Did you find this article interesting? Are you an “under the hood” kind of person? We’re really big on automation and we’re always looking for people in a similar vein to fill roles like this one as well as other roles here at Würth Phoenix.