What is wrong with Test Cases?
I have been reading various post on the topic "Breaking the Test Case Addiction" on the Developsense webpage. This is a great testing website and helps you to think about other ways to do testing.
To be honest, what caught my interest was the following text in the LinkedIn post by @Michael Bolton: "Even though software development has advanced enormously, testing is still stuck in many ways. One is the persistent fixation, visible in many places, on test cases."
Let get into this..
Software development has advanced enormously? When I read this, it surprises me we even talk about testing at all. If it has advanced so much, I would expect flawless software is delivered. The reality is the opposite. New languages, tools, methods in Software development have not really improved the quality.
Testing is still stuck in many ways? What surprises me also is that the testing community is always downplaying themselves. It seems testers have an inferiority complex against developers. I think that testing has helped a lot to improve development. Also I believe the testing community is very alive (blogs, events, gatherings). That is not exactely a sign of being stuck. Unfortunately, I see testing moving backwards since introduction of Agile and Devops in many organizations. There is a fixation on automation and developers doing testing. Little time and thought is being put into good testing.
One is the persistent fixation, visible in many places, on test cases? Ok, so the testing is still stuck, among others, due to fixation on test cases. In the article, it is explained that testing is a process and not only test cases. Is that new? No that is the first thing every tester learns when studying testing. Problem is that currently people get a training in JIRA and some automation tool and they call them tester or QA engineer straight away.
Although creating test cases (among others) helps to bring structure, it does not take away the creativity of the tester. The process of creating test cases includes investigating, questioning, exploring, reviewing, modeling, simulating, etc.
Maybe the issue are the test steps?
The real problem the article wants to address is, I believe, the fixation for detailed test steps (the script part of the test case). I can understand that part. In the beginning I was taught that anyone you picked from the street had to be able to execute the test steps. Luckily, we soon understood that was not achievable. Test steps are good to guide you while testing an application and can help to highlight some specific área that need special attention. But test steps should never be an obsession.
I personally do recommend you have an idea what you want to test. Some call that test cases, others test goals, others test scenarios. For me the main point here is that structuring your testing using test cases helps in doing that.
James Bach seems to have stated “testing shouldn’t be too focused… unless you want to miss lots of bugs.”. A tester needs to be able to see the detail without losing sight of the bigger picture. If you are not able to look outside of the box when testing, you are not a good tester. But a tester that is too loose will probably only report some corner case bugs. For me all testers should do some Jack of all Trades exercises to become a better tester. Try it!
Finally, there was a part on traceability. Test Cases can help to give insight into which requirement has been tested and which not. It does not say if it has been tested well or not. Also, it does not say the requirements were correct. But that is part of your testing process.
Conclusion.
- What is wrong with test cases? Absolutely nothing. When doing it right it is a great way to bring structure and approach your application.
- Should Test Cases be mandatory? No. As a tester you should have flexibility to approach your test object. But take into account your stakeholders. endusers, auditores and prepare your approach based upon that.
A test professional should be able to define a test approach based upon organization, applications to test, stakeholders, users, delivery process and date. Test Cases can help to bring structure, but are not the only way to do testing.
---------------------------------------------------------------------------------------------------------
Are you interested to learn more on defining a test approach for your organization without being tied to one specific methodology?
Please contact me by email: abeijleveld@test-cloud.com
Links:
- Developsense - Page os Michael Bolton
- OSSTMM Jack of all trades - PDF of this great execise by OSSTMM
- My top 10 testing books
- Test-Cloud
I dunno. I'm just simple Scrum Master but if a guy like Michael Bolton says that maybe there's a problem with fixation on test cases, then maybe it's something to consider. If I understand Mr. Bolton correctly, he's not saying test cases are not applicable. He's just saying they should not define testing. For example: Should a tester be crippled if every test case in their organization suddenly disappeared overnight? Of course not. In fact, I would expect an expert tester to be hardly inconvenienced at all. On the other hand, I have witnessed entire QA organizations become bogged down in the management of tens of thousands of test cases. In such organizations their reason for existing has become test case life support. In my experience, this situation is not uncommon. I believe it is this situation that Mr. Bolton is warning against. Test cases are NOT value. They are tools to achieve value. Therefore, create the absolute minimum number of test cases necessary to get the job done.
very nice! maybe I could disagree with you about the detail level on test cases, but in general lines I think you are correct. I someone state that "Testing is still stuck in many ways?", probably that person is not capable to just take a look around. congrat for this post Anko Beijleveld!