One of the biggest criticisms about testers and QA organizations is that they do not understand the business or the end-user. If you believe this to be true, it sends a clear message about not having a value-adding testing team of professionals. The more you know about the ultimate customer or end-user, the more you will become an effective, risked-based tester.
When I led QA teams in the past, I made "knowing your customer" a major performance criteria for my staff. To ensure this, I arranged field trips with business development to customer sites and had the testing team view how and why the end-users actually used the system or application. Upon returning from these field trips, the QA team started to modify how it approached end-to-end and user acceptance tests. It was so beneficial that the number of critical end-user defects dropped by more than 20 percent in a very short period of time.
This result inspired me to continue my learning. I took the product management certification course from Pragmatic Marketing and was certified in pragmatic product management in December 2009. From the course, I learned how understanding the following questions will increase the effectiveness of tests and testing teams (note: It is your responsibility to ensure you are adding value to the delivery of the product):
- What problem or problems will this upgrade, enhancement, or new feature solve? This is the value proposition.
- For whom do we solve that problem? This is the target market.
- How will we measure success? This is the business result. What metrics will be needed to validate success has been attained?
- What alternatives are out there? What is the competition doing? Is it a "blue ocean industry” or a "red ocean industry”?
- Why are we best suited to pursue this? What is our differentiator?
- Why now, and when does it need to be delivered? This is the market window or window of opportunity.
- How will we deploy this? What will be our deployment strategy?
- What is the preliminary estimated cost and benefit? Will there be a return on investment, customer satisfaction increase, or cost avoidance?
If you understand these high-level questions, you will ensure a higher level of end-user quality by designing and executing tests from an end-user’s perspective. Defining, quantifying, and weighing the many quality dimensions as perceived by your end-users, you will be able to approach testing in a very efficient and effective manner. Knowing what the user wants and needs to do with the system will enable a proactive mindset regarding requirements and feature reviews, acceptable behaviors, operational inconsistencies, interactions, and interoperability.
I have found the user manual to be a great source of knowledge for a test team. Granted, a newly developed application is devoid of a manual, as the manual gets developed along with the application. But, during my independent consulting years, I relied heavily on these manuals to gain an operational business perspective. Be careful, though, as they can be dated and may become stale depending upon how much the end-user relies upon them.
This perspective naturally leads to an understanding of where the potential risks are to the business:
- What are the most common and critical areas of the functionality from the user’s point of view?
- How accepting should the system be to incorrect input?
- Is the description complete enough to proceed to design, implement, and test the requirement processes?
- What is an acceptable response time for the users? Are there stated performance requirements?
- Which problems and risks may be associated with these requirements?
- Are there limitations in the software or hardware?
- Which omissions are assumed directly or indirectly?
About the author
Paul Fratellone is program director of quality and test consulting in the testing business unit of MindTree. Paul’s career of more than twenty-five years in information technology has been concentrated in testing, compliance, and quality management. He strives to achieve consistent execution to attain a predictable level of quality that is commensurate with the investment and enables leadership to objectively measure the success and continuous benefits from these investments.
