Evaluating Software Quality Tool Vendors
I have been involved in presenting and positioning and proving our software test automation solutions for some time now.
Oftentimes, the sequence of events is:
• Much effort is applied by the customer to produce an RFI or RFP which is issued to a list of test automation tool vendors
• Significant effort is applied by each vendor to respond with hopes of making the cut
• A selection is made by the customer based on some scoring about the suitability related to the responses
• The chosen vendor arrives on-site to do some sort of Proof of Concept
• If the chosen vendor works well enough, a decision is made to proceedThere are problems with this approach. The RFP or RFI that is issued is a hodge-podge of sundry, sometimes nebulous requirements that have little to do with real needs from a business perspective.
As you can imagine, these requirements are compiled by a larger group of people that each have their own agenda. The chance to see a solution in action is the prize at the end of the RFP or RFI process. This is the problem.
So much energy is spent defining the list of requirements and getting responses in textual form, that the core issue of “does the vendor have a solution that is effective and easy to use?” is lost in administrative silliness.
A more productive way would be to:
• Distill the needs of Software testing down to the core elements of what is needed
• Reach out to candidate vendors and define what these core needs are with a focus on what the business drivers are.
• Invite these vendors to come on-site and perform a focused, short duration Proof of Concept focused on the core issues
• Assuming more than one of the vendors were successful, optionally issue an RFP – a refined and extended list of requirements
• Make a decision based on first-hand experience coupled with an informed decision about suitability to your organization
Why is this better?
The proof is in the pudding.
Too often, a viable solution to core issues are never examined because of inappropriate agendas that have less to do with business issues and more to do with technical bias.
The path described above provides the needed reality check that the fundamental approach each vendor prescribes aligns well with what is really needed.
Without this, a decision is made too early and without the right focus.
