In this article, the author explains how to test when you're under severe time constraints and you need to immediately provide an assessment of the system's quality, and when there are no written requirements to test against.
The term System Functional Testing speaks for itself. The "system" means that the subject of testing in the whole software solution. "Functional" implies that we test behavior, visible to the end users. In some cases, system delivery to testing can be broken down into several stages. But even if a part of system is delivered to testing we still can approach it using system testing model. For the sake of clarity, we will not give another name to testing a part of the system.
System Functional Testing is verification of the system against behavioral and non‐behavioral product requirements. Behavioral (functional) requirements describe how system reacts to the user input or environment change. Non‐behavioral (non‐functional) requirements usually represent some property of the system, in contrary to behavioral requirements, which, by analogy with OOP, represent methods or functions of a system. Examples of non‐behavioral requirements:
- System must run on Windows 2000 (.NET 2.0).
- System must comply with ISOXXXXX security standard.
- System must run unattended 24x7 with mean time to failure no smaller than 1000 hours.
Know Your Mission
As any other process test design works best when approach on several level of abstraction. System test design works best when approached top‐down. The topmost level is defining the mission. Just ask yourself what the product is supposed to do for an end user. What services it should provide. What performance and system requirements are? Make sure you understand the requirements and your mission before defining a test strategy.
Do not hesitate to ask questions if something is not clear. Asking them at earlier stages is much better than waiting until it's too late.
Test Important Things First
Put the most important things first. Define what the users are supposed to do with the system first of all, find the most probably paths through the system functionality to generate your first tests. When you are out of wits try Help system.
Aside from the sequence tests contain associated data. As far as our goal is to check the system in conditions close to how it will work in user’s hands, we need to select typical data sets. Typical data is what users likely will submit to the system.
But be careful not to fall a victim of faulty assumptions on your side. For example, you may define a tiny database for a typical data whereas users may have huge databases. If you do not know for sure, just ask.
Usually critical path tests cover 80% of functionality with minimal number of tests.
Extend Your Testing
Testing of critical path allows quickly removing serious obstacles for a more in‐depth testing. But it's weak against issues that become visible at boundary conditions or erroneous cases. Extended testing is there to cover this potential gap in coverage.
Extended testing focuses on extreme values and values beyond the extreme (erroneous cases). It also requires thinking in terms of path permutation (identifying and covering multiple logical flows through the system).
Extend Your Testing Even More
There is something wrong with human brain. It fails to create complex models. The more complex the system, the higher is probability that you will learn something new about the system at test execution. This simply means that you may need to add tests once you tried the system out on the real system. If you noted that there is an interesting test that you would like to try but didn't have written, just do the test and put a quick note down somewhere to get back to it and transform in a test case later.
The whole idea of exploratory testing is grounded