Do Applications, Make Applications: Software Testing Approaches for Both

Not All Applications Are The Same

When we approach an application to test, we sometimes oversimplify it into a series of screens or tasks that the user performs to accomplish a goal. Before we review the application page by page, screen by screen, or prompt by prompt to determine how to test, we need to take a step back to determine what kind of application this is: Is this a Do application where the user makes some input into the computer and gets some sort of result, or is this a Make application, where the user does something different each time? You will want to test each differently.

For example, an online store or order taking application represents a Do application. The user, whether a customer or a customer service representative, picks a product to order, specifies a quantity, types in a shipping address and payment information, and receives confirmation. The workflow marches on from start to finish, with some variations that involve going backwards and forwards, allowing for some pre-population of saved information, and small detours into offering extended warranties where applicable.

A Make application, on the other hand, presents the user with a blank slate and a set of tools to create something. Sure, the user is doing something, such as writing a letter in a word processor, editing a photo, or designing a new and improved annular combustor for a jet engine in a Computer-Aided Design package. In this article, though, we’ll look at a bulletin board application that allows users to tack notes, files, and photos where others can see them. User workflow will be linear at a high level, but the user has a myriad of tools at any time to use in any combination and in any order of doing and undoing actions. 

When you look at Do or Make applications, you will want to alter your testing strategy to best test your application based on what the user does with it.

Exploratory Testing

Of course, you should perform exploratory testing on both types of applications; as a matter of fact, you can shake out a lot of defects during testing when you just mess with the software. Exploratory testing can give you a sense of what the application does (whether it Does or Makes) and where the code is fragile. All the planning and automation in the world cannot match the intelligence of an experienced tester clicking around and finding, even now, after the code is written, what might have been overlooked in design. 

Once you’ve done some exploratory testing with the Do application, you can come up with a pretty comprehensive set of test cases that cover much of the functionality of the application—and when you’re running the test cases, of course you’ll want to go a little off script to try different things and pay attention as you run the test cases as you mark step pass or fail. So you can allocate some time for exploratory testing and then for writing and running test cases.

With Make applications, though, you’ll want to give heavier weight to exploratory testing and allocate more time for it because, although you will want to create test cases as well, the test cases can really only capture at a high level the things a user can do with the application. So make sure you allocate plenty of time to get in there and click around using your own wits.

Straightforward Test Cases

With a Do application, you can capture a lot of expected behavior in regular test cases or checklists.

If we narrow down our online ordering example into just a Web site shopping cart, we find that the user has a series of actions he or she can perform to complete a purchase. You might capture those into a set of test cases like this in that sine qua non test case management tool Microsoft Excel:

This list only contains basic positive test cases. You should also test entering garbage into every edit box on the form, try ordering negative numbers of items, paying with expired credit cards, paying with debit cards that have insufficient funds on them, and so on and so forth. But these alternate test cases still are only branches in the tree, subtle variations in the workflow. And although the combinations seem infinite, they’re a smaller infinity than a Make application.

You can use the test case/test checklist model to capture the basic, high-level workflows of your Make application also. For example, with our bulletin board application, the user would have to be able to upload images, add text notations, and so on. So the list of test cases might look like this:

That could track closely with your list of requirements or acceptance criteria. However, the user of a Make application can do these actions in any order, and with the bulletin board application in a variety of states. Just passing through each test case only once won’t necessarily shake out the defects within the application. 

To better capture how to test a Make application, consider the State/Action matrix.

The State/Action Matrix

The State/Action Matrix differs from a mere list of test cases because, in addition to a list of cases to test, it also captures another axis that lists application states where the user can take the action.

The familiar test list appears on the left, but the top includes a list of states of the application where the user can try the action listed on the left. Some of the actions change the state of the application—changing the zoom level, for instance—but others perform more complex tasks.

You might be surprised, and possibly mortified, at how many issues you can identify when you try to group an image that was put into the trash and restored or what happens when you upload images at 4x zoom. The more tools your application offers the user, the more difficult it is to capture the larger infinity of user actions. However, the State/Action Matrix offers another dimension of testing to think about when planning and/or executing your exploratory tests.

When you’re running the tests in your State/Action Matrix, you will probably have the urge to work from the top to the bottom like you do with a list of test cases or an ordered list of instructions. You can further complicate your testing to match the complexity of your Make application by altering the order in which you run the tests. You might find different application behavior if you add a text notation before uploading and grouping a set of images, for example.

Conclusion

The Do versus Make dichotomy is a bit of an oversimplification. Most applications do a little of both, so you can think of it more as a continuum, with green-screen applications on the Do end and word processors, image and video processing tools, integrated development environments, and perhaps operating systems themselves on the Make end.

However, the terms offer a handy metaphor for assessing your applications and explaining why you need more time for exploratory testing on some applications than others. As the prince said, “There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy.” This is true enough for Do applications with fairly limited workflows. Make applications have morer things. The State/Action Matrix can help better capture some of those wildly divergent user paths to plan, to describe, and to document your test coverage. 

(Heading images via Pixabay's geralt and  Peggy_Marco.)