We're Building a Visual Regression Testing Library for React Native

It's called react-native-owl. It's open source and the first version with limited functionality is now available on NPM and GitHub at FormidableLabs/react-native-owl. The documentation can be found at https://formidable.com/open-source/react-native-owl/.

Introduction

So you've been building your app (whether that's a mobile app or web app), adding all the features over time, and looking to release it if you haven't already. But does it actually work as expected? How can we, as software engineers, confirm changes to our codebase haven't introduced any regressions? Writing tests that automatically test our code is a very important and ongoing process of software development. Depending on what you've been building you might choose different tools or even multiple ways of testing such as unit tests, end-to-end, integration tests, and, as you might have guessed it, visual regression is one of the many available options.

Visual example of a visual regression test

In case you are not familiar with visual regression testing, it is all about making sure the UI of your application hasn't unexpectedly changed. Imagine taking a screenshot of your application every time you make some changes to your codebase and then comparing it to the original one. Are there are any unexpected changes? Then your test suite will fail to prevent you from a "bad" merge.

Today we are announcing react-native-owl, a visual regression testing library focused on React Native. This is a library that enables developers to introduce visual regression tests to their apps for iOS and Android (react-native-web is not currently supported while expo support is just around the corner). Being heavily inspired by Detox, an end-to-end testing and automation framework, we were able to come up with a similar API that will make setting up react-native-owl and running the tests locally and on your preferred CI service, seamless.

Features

Creating such a library can be quite challenging as it comes with lots of moving parts such as a CLI, the actual library, the HTML report, and of course the native client that enables interacting with the simulator.

CLI

We did borrow the convention of our two main commands from Detox, having a quite similar CLI where you can build your mobile app and then run the tests against that build. Next up, the react-native-owl configuration file comes in place where you can specify the path to your React Native project or even provide your own custom commands if required. Running the build command will basically run xcodebuild and the relevant gradle command for Android. Finally, the test command will do some further preparation and then use Jest to run your tests for files that end with the owl extension (ie. App.owl.tsx), generating an HTML report that will visualize any differences if any of your test cases fail.

yarn owl build --platform ios
yarn owl test --platform ios

The core library

The library comes with an action (takeScreenshot) that enables taking a screenshot from the simulator while the app is running and stores it in the .owl/ directory along with the rest of the generated assets like baseline, latest and diff images, and the test run report.

As we are using Jest for the test runs, we also provide a custom matcher (.toMatchBaseline()) that will compare the screenshot with its baseline using Pixelmatch — a pixel-level image comparison library.

describe('App.tsx', () => {
  it('takes a screenshot of the home view', async () => {
    const screen = await takeScreenshot('homeview');

    expect(screen).toMatchBaseline();
  });
});

The Report

This would be the HTML report that will be generated if you have any failing tests and will show you a comparison between the baseline, the latest, and the diff image of these test cases. If you are running react-native-owl on your preferred CI, we suggest storing the output of the tests (.owl/) as an artifact so that you can download it if your tests fail. Then by viewing the generated report, you will be able to find the reason for the failure.

What's Next?

Building our library has been an exciting journey and we are looking forward to what's next as today we've only released a big portion of the ongoing work. Next, we will focus on building the native client, which will allow developers to interact with simulators, pressing on buttons, scrolling, typing, and more. We've seen the great work that has been done by Detox and we would love to build on it, utilizing it for all the aforementioned interactions with devices.