Why Visual Testing?

A core problem affecting web applications is visual drift due to code changes with unaccounted UI dependencies. In developing a product, small code changes can change the application visually in unanticipated ways. Writing assertions in E2E test frameworks to check every layout and style would be an excessive amount of work that would not have a positive return and would cause even more maintenance work down the line. This is where visual testing comes in to assert entire screens with a single line of code based on the screenshot and underlying DOM structure.

Before Astra:

The Challenge

Asserting an entire screen based on its screenshot it not a trivial task. A pixel by pixel comparison against a baseline screenshot are impracticable as small rendering changes across environment and browsers easily cause build failures. This is where computer vision algorithms come in to filter out irrelevant rendering changes.

The Astra check algorithm was trained on 100'000+ examples of irrelevant rendering differences, changing login times and further common differences that are irrelevant. With this AI driven approach, entire screens can be asserted with a single line of code and trigger build failures only upon truly relevant UI changes.

With Astra:

Besides the screenshot comparison, the Astra Check algorithm integrates the entire DOM structure into the check, providing you with relevant information about the root cause of the changes. The changes are highlighted on the screenshot in Astra making it easy to label changes, communicate them to your team and understand their context.

Results

Visually checking all critical screens in an application saves hundreds of hours of testing effort in a mid-sized application. The number of bugs slipping into production is typically more than halved, which reduces stressful hot-fixes on the latest release and reputational cost on the application users. This increase in application reliability allows development teams to ship faster with less effort.

Where Next?

Astra provides detailed labelling utilities that enables you to constantly gather new examples specific to your application. These new examples are trained into the algorithm on a daily basis to further reduce the number of false positives in your tests.

‍