Tradition To Cloud QA
Just 1 year ago, I joined this QA team to test our cloud products. I have been working as QA for traditional style of software development for years. There are so many differences between traditional QA and QA in cloud. Want to share my experiences.
Much More Short Release/Test Period
In a traditional software development, normally QA team has couple of months to run multiple iterations of tests. Also, we make sure there is no outstanding issue. All issues have been fixed or post-shipped before release.
When I joined the team, I was surprised by their two weeks release cycle. I was wondering how can QA team finish all tasks in two weeks. If counting from code complete to release, there is only a week or so.
But even in a traditional software development, we have similar situation as well, which is software maintenance release. A Hotfix or Service Pack or Hotfix Rollup Package contains one to hundreds fixes to the issues in current release version of the software. Software maintenance QA team doesn't have to and doesn't have enough time to run tests as the QA team working on new version software. Usually, they identify the impacted function areas from the source code changes. Only run full regression against the impacted function area, and cover other function areas by sanity level of tests, just make sure it is not broken.
New release in Cloud product is similar to software maintenance since developer don't rewrite everything. They add source code for new feature and update existing source code to fix customer issues. So, we can use similar approach as the software maintenance QA team in traditional software development. We have system to record source code changes from previous release, and what is the piece of code trying to do. Which is very similar as the maintenance team doing in traditional software development world. Based on that information, we can decide which area needs to be fully tested and which area just need a sanity testing.
New Source Code Affects All Customers Immediately
To deliver a traditional software to customers, usually, we test hundreds combination of install testing. We called it Packaging Test. Then the install package of software is deliver to web site, customers can download it by themselves. We may get issue reports from customers in couple of weeks after release. Also, issues in new version may not affect most of customers since they are waiting someone to "test" it. IT administrators always communicate with each other about the issues caused by new version of software.
In a Cloud product, new source code is delivered to Cloud, which basically a web site using by all customers. So, all customers will be impacted immediately. We don't need to worry about the Packaging Test here, but we need to deliver new version very carefully. There is a multiple steps release process. It takes weeks to deliver new version to all customers, and if any major issue is reported, we can always revert it back to old version anytime.
Automation! Automation! Automation!
Automation becomes such important today since computer system becomes more complex, cost of manual testing becomes higher and higher. Everybody is talking about how to automate their tests. We are not exception, of course.
Even if great amount of efforts have been leveraged to automation. There are still lots of space to improve. One big area is collaboration between automation and manual testing teams. Air power alone can't win war. Infantries are still necessary in modern war.
