Testing Modern Web-Apps
Web apps/sites today don’t look the way they used to a decade ago. In fact, they look very different from what they looked 5 years ago. With javascript frameworks such as AngularJS, ReactJS, Backbone/Ember as well as server frameworks such as Django, Flask, Node, Rails the way you test your web site/app has changed as well.
The purpose of this post is to:
1. Think about a decent organization of the various kinds of tests for your web app/site
2. Think about the pros and cons of using the various combinations of tests
3. Present one (of several) ways to organize various tests for a modern web-app
The diagram posted on top shows a typical simple web-app. Our goal is to test the various parts of the above diagram, either as standalone units or as a whole (or a combination thereof) with the least amount of redundant testing.
There are various different tests you could have in such a system. I will consider a system using:
1. AngularJS + jQuery front end
2. Django Back end
But the analysis here should be easily applicable to a different front end (such as ReactJS or EmberJS) or back end (such as Rails or Flask or Node).
The different kinds of tests in such as system could be:
1. AngularJS Unit tests
2. JavaScript module unit tests
3. Django unit tests
4. Unit tests for standalone Python Libraries (used in the server)
5. End to End tests (using Selenium for example)
We need to make a decision whether or not to support each of these kinds of tests, how much weight to give to each and how to measure the coverage for each of these.
Let’s think about what each of these tests are testing, and what they are not testing.
1. AngularJS Unit tests - These tests are testing your Angular Components such as directives, controllers, services in a standalone fashion (as if these components exist outside of the overall system). In short, they are testing some but not all components that run inside the browser in the above diagram. Also remember that there may be pages that aren’t completely or perfectly componentized such as those using 3rd party libraries such as jQuery plugins.
2. JavaScript Module unit tests - Assuming that you have modularized most of your JavaScript code (which is highly recommended), you can test these modules as standalone components. These tests fall in the same category as (1) that is AngularJS Unit tests as by testing these you will be testing certain parts of the system that run in the browser, in a standalone fashion.
3. Django Unit Tests - Django provides a unit test runner that can invoke Django views in the same way as a browser client would. There are actually 2 ways to use these tests.
3.1 You could mock out your db access part (for example if usually your view retrieves data from a DB, you could substitute with a mock api that either returns hard-coded data or data from a file system. In this case, you will not be testing the circuit labeled (3) in the diagram above.
3.2 You could test the full circuit against a test DB such that the circuit labeled (3) is also tested via your Django unit tests.
4. Unit tests for standalone python libraries - Surely your Django views must be using other standalone Python libraries and you should have separate unit tests for these components, especially since these components could very well be used outside of the Django environment by other (possibly non-web) applications.
5. End to End tests - Finally, web-apps these days are commonly relying on test frameworks such as selenium to invoke end to end requests in a headless or browser based environment to test the full circuit.
These tests are great as they test the entire system but come with a few problems:
- In my experience modern rich UIs aren’t as easy to test with Selenium as the headless browser isn’t always up to date with features you may be using
- These tests are time-consuming, to design as well as to run. To design you need to record user requests via UI interaction. Running these tests invokes the full circuit (from browser to db and back) and need all parts of the system to be up and running at all times)
- There is some level of redundancy here since in each test you would end up testing the circuits 1, 2 and 3 in the diagrams above and there may not be a huge benefit to that - the argument being that if the data packaging, serialization to JSON etc. and deserialization is working for Page 1 then it probably doesn’t need to be tested for Page 2.
It should be clear that to test our system in a practical and thorough way, we need to use a combination of these tests. Using only End to End tests is not a feasible solution, as it ends up being very time consuming to write as well as to run, and that could server as a deterrent to writing more tests and running them often.
The big question is, what is the right combination of these tests. There could be several opinions and answers to that, and each system and team is different in it’s needs and environment. One way to combine these tests is presented here without claiming that this is the best or perfect way to do so:
A Possible Organization
The summary of the method is to use all of these tests together with different weights given to each. The idea is to test most of the core logic using various unit tests and have only a few end to end tests. To elaborate:
1. Use AngularJS and JavaScript unit tests (1 and 2) using a tool such as Karma/Jasmine/Mocha and measure coverage using a coverage reporter. Test each JavaScript module.
2. Write Django unit tests for each view and test as many code paths as you can and measure coverage using coverage.py . I recommend using a test database as opposed to mocking out the DAOs. This is especially better if you are using something like SQL Alchemy or Mongo QL since you would then end up testing the logic of your DAOs as well. This obviously is not an absolute rule and there could and would be exceptions.
3. Unit test your standalone Python (or Java for that matter) libraries.
4. End to End tests: Use a tool such as Selenium to write end to end tests BUT use this in a broad way. The goal is to ensure that:
- The overall circuit is working
- The pages that aren’t well componentized are tested
For example, you could have tests for:
- Login, Logout (Authentication)
- Have at least 1 end to end test for each page.
- For example if you have a lot of search pages then invoke the search for each page, but don’t test ALL the possible combinations of input search fields as the Django Unit tests should be testing those.
- If you have updates and insert then it is recommended to have a test for the update as well as the insert testing from the point the users fills a form and presses a button to getting a response back and verifying that the data was actually updated/inserted.
Remember that end to end tests are very different from unit tests for the front end as unit tests (whether AngularJS or JavaScript) are only testing a single or a few components rather than the interaction of various components (jQuery, AngularJS) with each other and with the DOM.
I have not covered event or pub/sub related tests such as for Celery, RabbitMQ related components but you should definitely have some end to end and many unit tests for the same.
The overarching summary of this post is:
1. Write many unit tests on each tier of the stack and try to make them run fast
2. Write some end to end tests since they slow the system down, relying on the fact that you have thoroughly tested the core logic of the system via unit tests.
I hope this provides a line of thinking towards testing a modern web-app. This isn’t meant to be definitive or the only way to organize your test stack, but is one method.
Comments are welcome.
References: