Ember-Backstop Visual Regression Testing Tutorial
Learn how to add visual regression tests to your Ember app in 10 minutes -- with the Ember-Backstop test-helper addon.
When you do a UI code review, how do you know that the CSS and markup your merging looks the way it should? Do you simply take the word of the author? For a lot of web-app developers the answer to this is a deflated yes. But for many Ember developers, that answer may be about to change!
The BackstopJS team has released a free, open-source, visual-test-helper addon for EmberJS. It's called ember-backstop and it magically enables adding powerful visual tests to your ember app. This tutorial will walk you through it.
Setting up
The demo repository is here so you can download it and follow along.
To create this demo, we started with a fork of the EmberJS Super Rentals tutorial and configured it to use the ember-backstop test helper -- so you can get right in and experience visual testing with everything all ready to go!
To start, make sure you have the following things installed on your machine... Git, Node.js, Yarn, Ember CLI, Google Chrome.
Then, install the demo. It's fine to install it on your desktop or wherever you keep your experiments.
git clone https://github.com/garris/ember-backstop-tutorial/ cd ember-backstop-tutorial yarn install
Start Backstop-Remote Service and Ember Server
From the current project root, run the following to start the Backstop-Remote service.
ember backstop-remote
Then, in another window (again from the current project root) start ember.
ember serve
If everything worked your server will be running and the demo app will be hittable at http://localhost:4200. Go ahead and click around there. We'll wait...
All Good? Ok. let's check out the tests!
Reviewing the Conventional Tests
Conventional Ember tests (Acceptance, Integration, Unit, Linter) are already set up. Go ahead, hit http://localhost:4200/tests and watch everything run. Here is the testem test report that you should see.
Next, set-up the ember-backstop test helper
First, open the file at `tests/acceptance/list-rentals-test.js` and add the following import statement to the top of the file.
import backstop from 'ember-backstop/test-support/backstop';
With the helper inported, here is an example of how you add the backstop() helper to your tests...
test('it renders the thing', async function(assert) {
await visit('/sales/company/11102');
await backstop(assert);
assert.dom('#myFancyElement').Exists();
});
Notice the `await backstop(assert)` helper in the code sample above (on the third line). The backstop() helper serializes a snapshot of your DOM at that moment in time and posts it to the Backstop-Remote service. From there BackstopJS is invoked in a dynamic test flow.
An aside about dynamic testing with BackstopJS
Dynamic testing is a BackstopJS feature (added in mid-2019) which allows you to create an arbitrary collection of visual regression tests attached to any unique test-run by a host test framework. (If you are familiar with BackstopJS, then you can consider this a way of generating config scenarios on-the-fly). So, together with the backstop() helper, each time you run a test in Ember, regardless of whether you use the browser version or the `ember test` CLI method, BackstopJS will be invoked, generate test screenshots, compare them to thier reference screenshots (if they exist) and return a pass or fail response right in-line with your existing test flows. In our experience, the visual tests are reasonably fast -- adding 1-3 seconds per visual test. Now, back to more testing fun...
Adding `backstop()` statements to your tests
So, go ahead and add the backstop() helper to the last four tests inside `tests/acceptance/list-rentals-test.js` as shown in the below figure.
Notice the `await backstop(assert)` statements in lines 43, 49, 57 and 65 outlined in red above. That's what you should add!
Running ember-backstop tests for the first time!
The first time you run an ember-backstop test it will fail -- so go ahead! Fail fast! Rerun your testem tests (refresh http://localhost:4200/tests) -- it should now look like this...
Testem is showing failed BackstopJS tests -- these failed because we have not yet told BackstopJS what to use for reference images. Let's fix that! Open the BackstopJS report...
ember backstop-report
That report should look like this.
Here we see that screenshots were generated correctly and the screens all check out -- so, let's tell BackstopJS that we approve! This will remove those error messages and tell BackstopJS to compare future screenshots against these.
In another window, from the project root, approve the tests with...
ember backstop-approve
Now run tests again (refresh http://localhost:4200/tests) -- they should all pass!
When the testem tests have completed go ahead and open the Backstop report as well `ember backstop-report`. It should look like this...
Congratulations! BackstopJS tests are all set up!
Great! So, whats next?
Just add the backstop() helper anywhere throughout your tests -- be strategic -- because -- just like any other kind of test -- maintenance can be a thing.
For a little more practice while we're here
As a suggestion, you could open up the project's CSS file at `vendor/ember-tutorial.css`. Try deleting the `postion: static;` as shown below. Refresh at http://localhost:4200/tests and see if BackstopJS can find the changes... 😉
The BackstopJS report should look like this.
Try it in your own Ember App!
Because it's part of the Ember ecosystem, installation is super-easy and quick. To install ember-backstop into your own project run the following...
ember install ember-backstop
Detailed install & usage doc here https://github.com/garris/ember-backstop
Further Reading / Useful Links
Connect with me on Twitter or LinkedIn!
@garris -or- https://www.garudax.id/in/garrisshipon/
☮️💜
Ember, the Ember logo design and the Tomster designs are exclusive trademarks registered in the United States by Tilde Inc.. Used here with permission.