handles an error case.” (Marick)
Test Case – a Product and its Quality
Above discussion now takes us to the next point in our journey to understand test case as a product and quality of this product. First, let us see what quality, in general, means to us.
Quality is defined as-
Fitness for use (Dr. Joseph M. Juran)
Conformance with requirements (Philip Crosby)
Quality is value to some person (Gerald Weinberg)
Detailed discussion about above definitions is out of scope of this paper but these definitions form a good base for defining software product quality.
I tend to say that software product quality is a multi-dimensional measure of meeting requirements of users at an affordable cost.
If we apply above statement to quality of a test case as a product, we could say that it is, multi-dimensional measure of meeting requirements of information objectives at an affordable efforts and cost.
It is extremely difficult to draw a line through all aspects discussed above and come out with a central idea of quality of test case as a product. The practices that we evolve at our organization address our day-to-day needs of meeting the test project objectives and consistently satisfy our stakeholders. However, in the next section, I have attempted sharing my thoughts about how can we engineer test design activity in such a way that the end product is of high quality.
Test Case Engineering
If we call a program as a function of data and operations on data, we treat a test case as function of carefully chosen set of data for the purpose of operations on it for a given information objective.
Test Case Engineering (TCE) starts by breaking down the software product into the three fundamental issues – the Technologies used, the Business domain (or the problem domain), and the Architecture of the product.
Figure 2: Product under Test
Once TCE identifies the three issues well, it grips the whole product with the help of practices in TCE that are depicted in the model below. The practices that one builds in an organization form the Test Case Engineering as a predictive activity to deliver expected quality of test cases.
Figure 3: TCE Practices
Above model clarifies how TCE is a careful thought process to design test cases that will encompass three distinct issues of any software product. The practices employed can be divided into following groups-
- Test management- leadership
- Knowledge Management
- Test designing practices
- Identifying information objectives
- Choosing test data
- Mapping to
- Development model
- Test types
- Test execution techniques- Manual or Automated testing
- Test methodologies- Functional or non-functional testing, White, Gray or Black-box testing
- Test case quality management processes
- Context-driven Reviews
- Test case management system
- Test case version control
TCE helps test teams achieve a balance between expectations, time in hand and quality of work. It helps them decide how and when to trade-off without compromising the objectives of the testing.
Each of the practices described above needs a self-correction mechanism in place. The discussion about the self-correction mechanism is beyond the scope of this paper.
The systematic approach of TCE can deliver high-quality test cases inherently and consistently if the test organization treats test cases as their product and apply the notion of quality to this very important aspect of the entire test management processes.
Dr. Cem Kaner about ‘Good Test cases’
In the article ‘Good Test Case’, Cem Kaner discusses characteristics of good test cases. (This article can be downloaded from kaner.com).
Cem summarizes characteristics of a ‘good test case’ to begin with-
Designing good test cases is a