Test Automation with Javascript based tools, libs & frameworks

Test Automation with Javascript based tools, libs & frameworks matrix by features :

As JavaScript automated testing frameworks have matured below have risen as the most popular and can be categorized by their protocols:

1. World Wide Web Consortium (W3C) WebDriver – WebDriverIO and Nightwatch
2. Chrome DevTools Protocol (CDP) – Puppeteer and Playwright
3. Proprietary – Jest, TestCafe and Cypress

Jest is the testing framework created and maintained by Facebook. It spiked in popularity and became the most popular testing library in 2017 and stayed in the first place since then.

It was initially based on Jasmine which I will discuss later. Over time, Facebook replaced most of its functionality and added a lot of features on top of it.

• Performance- First of all Jest is considered to be faster for big projects with many test files by implementing a clever parallel testing mechanism.
• UI- Clear and convenient.
• Ready-To-Go- Comes with assertions, spies, and mocks that are equivalent to libraries that do the same like Sinon. Libraries still can easily be used in case you need some unique features.
• Globals- Like in Jasmine, it creates test globals by default so there is no need to require them. This can be considered bad since it makes your tests less flexible and less controllable, but in most cases it just makes your life easi
• Snapshot testing- jest-snapshot is developed and maintained by Facebook, although it can be used in almost any other framework as part of the framework’s integration of the tool or by using the right plugins.
• Great modules mocking- Jest provides you with an easy way to mock heavy modules to improve testing speed. For example a service can be mocked to resolve a promise instead of making a network request.
• Code coverage- Includes a powerful and fast built-in code coverage tool that is based on Istanbul.
• Reliability- Since it has a huge community, and used in many very complex projects, is is considered very reliable.
• Support- It is currently supported by all the major IDEs and tools.
• Development- jest only updates the files updated, so tests are running very fast in watch mode.

Cypress is a direct competitor of TestCafe. They are doing relatively the same, which is injecting tests into a website, but they try to do it in a more modern, flexible and convenient way.

The difference between them is that Cypress.io runs itself in the browser and controls your tests from there where TestCafe runs in Node.js and controls the tests through a serialized communication with its injected script in the browser.

Parallel testing was introduced in version 3.10.

• Documentation- Solid and clear.
• Native access to all your application’s variables without serialization (TestCafe on the other hand turns objects to JSON, sends them to Node.js as text and then parses them back to objects).
• Very convenient running and debugging tools- Easy debugging and logging of the test process.
• Cross-browser Support- since version 4.0.
• Some use-cases are missing but in constant development such as lack of HTML5 drag-n-drop.
• Using Mocha as the test structure provider makes its use pretty standard and allow your E2E tests to be built in the same structure as the rest of your tests.

TestCafe is a great alternative to Selenium-Based tools. It was rewritten and open-sourced at the end of 2016.

TestCafe had also a paid version that offered non-programming testing tools. It has been deprecated and got replaced by the new TestCafe Studio.

TestCafe injects itself into the website as JavaScript scripts instead of controlling the browsers themselves like Selenium does. This allows it to run on any browser, including on mobile devices, and have full control over the JavaScript execution loop.

• Fast to set up- Npm install and run your first test on any browser you want.
• Cross Browser and Devices- Supports many browsers and devices and can be used with SauceLabs or BrowserStack which provide devices and browsers for your tests. This includes running of tests in Headless Chrome and Headless Firefox which will be discussed later.
• Parallel Testing- TestCafe can run your tests on several browser instances at once. This practice can significantly decrease your testing time.
• Convenient Error Reporting
• Own Ecosystem- TestCafe uses its own test structure. It could be quite convenient, especially because UI tests usually run separately from other tests, but some people dislike it.

Puppeteer is a Node.js library, developed by Google. It provides a convenient Node.js API to control Chrome or Headless Chrome through an API the browser exposes: the DevTools Protocol.

Headless Chrome is just a regular Chrome v59+ that is launched with the

--headless flag. When Chrome is run in headless mode, it exposes an API to control it, and as said before, Puppeteer is the JavaScript tool that Google provides to control it.

Notice that different testing tools can also use Headless Chrome and Firefox. For example TestCafe, Karma, and Cypress.

• Puppeteer is maintained by Google and has a big community that uses and develops tools and wrappers around it.
• Since it is native and uses the latest Chrome engine, it is very fast.
• One major drawback of Headless Chrome (thus of Puppeteer as well) is that it doesn’t support extensions like Flash and probably won’t in the near future.
• Notice that this tool doesn’t come with a build-in testing system. Tools like mocha and chai can be used to create the necessary testing environment and structure around this library.

Playwright is like Puppeteer extended to more browsers. It is developed by Microsoft by the team that originally developed Puppeteer.

The library automates Chromium, Firefox, and WebKit (Safari) by using the DevTools Protocol for Chromium, and similar technologies for other browsers.

It’s worth mentioning that since the library is pretty fresh (It was released in January 2020), breaking changes may be made in the near future and some things might not work or be documented as expected at the moment.

• Cross Platform
• Supports multiple tabs
• Pretty new

Conclusion

JavaScript automated test ecosystem has evolved alongside it to better suit the needs of modern web application development. Currently, three general approaches have solidified:

1. Using standards such as WebDriver
2. Using non-standard APIs for each browser, such as CDP
3. Using proprietary approaches such as proxy servers and pure Node code

Regardless of the technique used, each JavaScript automated test framework has its pros and cons. The better we understand where these frameworks/tools excels and falls short, the more equipped we are to select the test framework that best fits our environment and context.

Javascript tools learning path : It is a nice learning path any one can explore:

https://testautomationu.applitools.com/learningpaths.html?id=web-ui-javascript-pat