identified cases. With a product design that has no testability in mind, this becomes challenging.
Some kinds of behaviours are notoriously difficult to reproduce and hence difficult to test. They include behaviours that takes time, behaviours are non-deterministic (or so it seems), behaviours that go across threads, processes and machines, behaviours that involve hardware, behaviours that involve large data sets, behaviours that involve concurrency, synchronization, and locking, asynchronous behaviours and more.
When coaching teams about testing, one of the first thing I do is to evaluate the products testability and strive towards what I call "Testability Coverage". This means identifying the different kinds of tests and enhancing the product and its test environment tool cover all these kinds of tests. It is only after testability coverage is achieved that you can write automated tests and there after you can consider how test data coverage can be achieved.
Testability is a quality of the product that measures how easy it is to test it:
- How easy it is to control the behaviour of the product - controllability
- How easy it is to observe the behaviour of the product - observe-ability
- How easy it is to isolate the problem if one is detected - diagnose-ability
Testability is also a quality of the test cases
- How easy it is to prepare a test case?
- How easy it is to prepare the test data, test scenario/script?
- How quickly can each test case run?
- How robust are test cases to minor changes and noise in the test environment?
Testability is also the organization of test cases and test results?
- How organized are the test cases? Can they be found easily?
- How easy it is to evaluate test coverage (from various perspectives)?
- How easy it is to interpret the results of test execution?
- How easy to infer the quality of the system from test results?
As you can see, testing is a challenging endeavour. I do encourage true developers (people who love code) to start thinking like real testers, to explore the test data and to build testability into their products and test environment. When developers love to test and put on their thinking (testing) hat, you will see a remarkable change in your product's quality. I get an indescribable sense of pleasure when I see developers write their own tests and they get it. They do not complain, they welcome the new challenge. It is like an ocean of knowledge opening up for them to devour. They ask questions that challenge me and we solve them together. This is the kind of coaching I like to do as opposed to playing the role of a mother who nags and says “please test”.
5. I motivate this change by getting the developers to look at the results of code coverage tools. These tools show which line of code is not executed, and encourage the developers to think why? And later think about how to control code behaviour to execute the line they want. This also helps developers write better code, which is also a very important aspect of quality. This is not something a typical naive tester is capable of. Mindset: Effective Separation of Concerns
I have earlier mentioned about the debates between proponents of big up-front design and versus those of emergent design (which is sometimes mistaken as no up-front design at all). Of course, I believe some initial analysis and design is necessary and this also has to be followed by regularly revisiting the design for further improvement.
But whatever your stand is, you want to eventually get a good design. So, what is