It is time for a new build. What should be included in it? Obviously, it should include the latest and greatest versions of each module. Right?
The phrase "latest and greatest" belies an assumption that the latest version of something is automatically the best. The latest version adds features, corrects problems and, in short, improves on the versions that preceded it. How could it be anything other than the greatest?
Things are often not as great as they seem to be, unfortunately. Those new features may be incompatible with other existing functionality and things that users previously relied upon may have disappeared. The new features may impair usability, particularly for novice users. There are also bugs that invariably crop up in all of that new and changed code.
So, how can we determine when the latest actually is greatest? How can we know when the code really is ready to be included in the next build? Many development groups solve this problem by establishing promotion criteria. Promotion criteria are policies about how a particular module's readiness for inclusion in a build is determined.
Unit Testing Standards
Though there are many different things that could be included in your promotion criteria, unit testing is the foundation on which they are all built. Almost every organization assumes that software developers are performing appropriate unit tests. Unfortunately, different individuals tend to have radically different ideas about what is an appropriate amount of testing.
A good practice is to require that the developer document the tests he or she will run, and have peers review those tests to ensure appropriate coverage. If automated testing is used, then the developer can simply create the test scripts for the automated tool, and submit those scripts for review.
Of course, establishing group standards on what should be included in the unit tests is also a must. Coming to an agreement as a development group about what testing should be done will take some time and deliberation, but the time spent on this will be paid back many times over in builds that go right! Let's look at some examples of unit test expectations.
Each module must be tested to ensure it satisfies its design and actually does what it should do correctly. What inputs must it handle? What things must it do? What services will it provide? What outputs should it produce? What data must it manage, and what must it do with that data? We must ensure that the module actually does what it was intended to do. Negative Testing
Then there's error handling. Does the module do the right thing when things go wrong? What happens when it is presented with invalid inputs? What if they are poorly formed or out of sequence? How about non-numeric data when numbers are expected? Data overflow? Underflow? What if it gets an error status back from the database or the network interface? What does it do then? The module must handle all error conditions correctly before it is considered to be complete.
We all know that exhaustive testing of software is not a reasonable goal. There are simply too many combinations of inputs, too many orders in which events can happen, and too many different ways things can go wrong to be able to completely test everything in even the most trivial program.Code and path coverage, though, are an achievable goal for unit testing. In fact, unit testing is the only time that complete coverage of code and paths is a reasonable goal.
It’s reasonable to require that every line of code be executed during unit test, (and analyzers exist to help with ensuring that this is done). Some code (especially error handlers) cannot be tested without taking extraordinary steps