Do Your IT Projects Suffer from Requirements Clarity Issues?
A common cause of IT project time and cost overrun is the impact of requirements issues on systems, where the expected system behavior is not clear. Software testing is based on the underlying "oracle assumption," a practice in which testers routinely predict the behavior of the system being tested based on written requirements, old systems, or business representatives. However, the oracle assumption sometimes breaks down due to project pressures or system complexities. Common issues that occur with requirements include:

• They are too vague and open to interpretation or contain insufficient detail for creation of the required tests.
• They contain gaps where system behavior is unspecified.
• They lack clarity about exactly which requirements are to be delivered in the release to be tested.
• Decisions made about requirements in earlier stages (e.g., clarifications obtained by developers) are not stored centrally and not passed on to later phases, including testing.

Specific examples of requirements issues include lack of exception handling, including recovery actions for data load failures; dealing with incomplete input data; and handling special cases in calculations.

It is well known, yet frequently underestimated, that the later in the project that issues are uncovered, the longer and more expensive they are to resolve. This is due to the increased amount of rework required to correct defects, which involves more people with each completed step in the development process. Surprisingly, late discovery of issues happens even on projects that are following a lifecycle with signed-off requirements and change-control processes. In my experience, the larger and longer the project, more significant the issues uncovered late in the cycle become.

You may wonder how much impact requirements issues have on a project. Here is an example from two large systems that I worked on recently:

• Ten percent of the issues found during acceptance testing were related to requirements.
• Those issues alone took the project over 1000 man-days to resolve.
• The other 90 percent of UAT (User Acceptance Tests) issues, as well as all of the system-testing issues, also had to be resolved, which created a huge impact on overall timescales.
• The requirements issues on a second system were quicker to resolve, but there were more, resulting in an approximate 300-man-day impact.

Requirements Issues Found During Acceptance Test
The impact that this can have on the project timescales can be considerable and includes:

• An increase in the effort spent getting clarifications
• An increase in the effort spent investigating, fixing, and retesting resulting defects
• An increase in the number of test-fix-retest cycles needed
• Resources tied up in rework and taken away from other tasks

You could argue that finding these gaps is exactly what testing is for. However, initial project plans created based on the amount of testing effort required in the above examples would be thrown out as unrealistic.

What Can be Done to Address Requirements Issues?
Prevention of extensive requirements issues in the above examples would have been desirable. Even if the preventative measures were only 50% effective, they could have saved 150 to 500 man-days' effort in the example projects.

So what could be done to prevent requirements issues? Projects in the example above use a modified waterfall software development methodology, which includes documented requirements and some document reviews. However, some of the reasons that the requirements issues were not caught earlier include:

• There were no requirement content checklists available to reviewers. These help to confirm completeness of the requirements by giving pointers to requirement content rather than structure, to help prevent key areas being missed.
• Testers and end-users did not always get the opportunity to review and give feedback on the requirements before they were implemented.

A suggested approach is:

• Create a good checklist to use for reviewing requirements for completeness.
• Have business analysts, designers, and developers use requirements checklists throughout the project to help prevent requirements issues.
• Involve testers and end-users in reviews of requirements and designs throughout the project. Although this takes time, it prevents multiplication of errors by catching them before they are developed and also builds test team members' knowledge of the system, making them effective when they need to start building test cases--not playing catch-up and asking basic questions about the requirements at that stage.
• Add system specifics to the requirements-content-review checklists as they become known (e.g., specifics of overnight processing, your functionality), so they can be used in maintenance and regression testing for future versions.

Curing Requirements Issues
What should you do if you're in a situation where you've already got requirements issues? Suggested steps are:

• Use a requirements content checklist to identify requirements issues as soon as you get the requirements. It takes time and effort to do this, but repays later.
• Raise defects or queries (whatever your project process is) against the issues as soon as possible.
• Collect specific examples of the issues you have and metrics on the effort involved in resolving them.
• Try to predict the impact of the issues on the project timescales. You probably won't be allowed to extend the timescales for a project, but it's important to communicate the likely impacts to appropriate stakeholders.
• Identify a central place where requirements clarifications will be stored, so effort is not duplicated in later phases of the project. This could be a change-control system, a folder on a shared server, a public email box, or a wiki.
• Share requirements clarifications with later phases of the project, new starter training, support/maintenance, and regression test efforts.

What do You Put in Your Requirements Content Checklist?
No single requirements checklist can be used for all systems. The following checklist contains some starting points which can be used as a basis for content review, although it is not exhaustive. The best checklists also contain technology platform and system specifics.

Have requirements been specified for the following areas?:

All Front End User Functions such as menu options, screens and processes which implement the main functional requirements for the system.

• Confirm that front end user functions cover all the tasks that the user must perform
  
  • Compare the against a model of the user process, such as a business process description
  • If different types of user role are involved, do the functions fully cover all roles e.g.) shopper, finance, customer services, IT support
  • Do the requirements specify whether data insert, update, delete and find actions are valid for each function and user role?

Administration Functions which allow management of system settings, to assist the system's primary functions. For example, a screen to set up user permissions.

• Typical areas for administration functions include:
  
  • User permissions, passwords and other security settings
  • Picklist content maintenance and user visible options.
  • System parameters including processing settings, paths and locations, size limits
  • Settings for audit trail, backups

• For each administration function, are the following specified?:
  
  • Default values for settings
  • Valid combinations of setting value
  • Assumed initial live values

• Some administration functions may not be implemented in the system - some will be carried out manually by an experienced system administrator:
  
  • Which administration functions will not be implemented?
  • Are assumptions about how any manual administration functions will take place stated?

Housekeeping Functions which check or maintain the integrity of the data processing, to assist with system support. For example a check which generates an email notifying whether a data transfer has completed successfully.

• Typical areas where housekeeping functions may be required are:
  
  • Data import/exports
  • Batch processing
  • Data synchronization
  • Periodic data archival

• For each housekeeping function, are the following specified?:
  
  • Frequency of function being run
  • Any notification of errors/warnings
  • Actions on warnings/errors

Data validation checks which confirm whether data entered, updated and deleted in the system meets required rules.

• Typical areas where data validation checks are required are:
  
  • User inserted, updated, deleted data
  • Data imports/exports
  • System generated, updated, deleted data

• For each area of the system where data is validated, are the following specified?:
  
  • Valid values/ranges
  • Default values
  • Whether nulls allowed
  • Special characters allowed e.g.) punctuation
  • Calculations/derived values
  • Picklist/default values
  • Combinations of values allowed
  • Referential integrity checks on deletion
  • Audit trail requirements
  • Error handling/action if data is invalid

Business rules include special cases of system processing which are triggered only in certain system conditions or by certain data combinations. Business rules are not always fully articulated in requirements because they are sometimes assumed to be 'obvious' or are dependent on dynamic system conditions.

• Typical areas where business rules may exist include:
  
  • Where data entered into the system means that from that point forward a specific subset of processing must apply e.g.) on a shopping website checkout page, choosing a 'credit' payment card type affects the required data entry fields and the type of anti-fraud checks available.
  • Where business decisions mean only certain values of data will be used, even though the system will permit a wider range of values. E.g.) In a travel industry system, the business decides to divide travel into packages of 3, 4, 7, 10 or 11 night durations. If you want to buy 14 nights of travel, the system must actually allocate you two 7 night packages.

• For each business rule, are the following specified?:
  
  • Values/combinations of values that trigger rule specific processing
  • Subsequent processing tasks and life-cycle of valid states
  • Exception/error handling

Data import/exports where data is exchanged with other internal or external systems.

• For each import/export are the following specified?:
  
  • Data specifications including valid input/output values, defaults, nulls
  • Handling of invalid values
  • Triggers for the import/export
  • Processing and calculation rules
  • Logging, correction and reload
  • Expected volumes; temporary space required; upload times and frequent
  • Checks for success; recovery from failure
  • Transport method used e.g.) FTP
  • Data source system availability and integrity checks

Batch/system processing functions where processing takes place on a periodic basis and may be triggered by a schedule within the system.

• Typical areas where batch/system processing functions occur include:
  
  • Synchronization of data between locations
  • Data imports/exports and subsequent processing functions
  • Periodic tasks such as report creation, forecast generation

• For each batch/system processing function, are the following specified?:
  
  • Triggers, frequency
  • Assumed live schedule of tasks, processing time windows e.g.) 5-8am
  • Checking /notification of failures, recovery from failures - minimum time to get back up to date if fails, for how long does it matter if job doesn't run
  • Optional tasks, special days (weekends, last day of month etc)

Reports where the system outputs data to screen or hard copy

• For each report are the following specified?:
  • Layout/column order, formatting
  • Calculations/processing
  • User customizations available
  • Ad-hoc/batch run schedule
  • Processing requirements - frequency, duration, special days (Mondays, weekends, last day of month, etc?

Modes of operation where specific features are built into the system to enable or disable certain sets of functionality. Modes of operation are sometimes set up to protect users or substitute for functions that are not available in certain circumstances.

• Typical areas for modes of operation include:
  
  • Restricted modes which offer cut down sets of functionality
  • Test modes which prevent test data being sent to external areas or simulate these areas which are unavailable for testing e.g.) live trading system
  • Fallback modes which allow user processing to continue in a limited form even if a key external system is not available
• For each mode of operation, do requirements specify:
  • How the mode is set up; how to check it is working
  • Impact/restrictions on system
  • Life-cycle of processes in the mode?

Non-functional requirements - typically no specific functionality is implemented to support non-functional requirements, but the system overall must meet certain criteria

• Typical areas of non-functional requirements include:
  
  • Performance - how the system should respond in normal and exception circumstances
  • Data volumes - predicted database size and transaction numbers over time
  • Security - resistance to unauthorized/malicious access
  • Usability - ease and effectiveness system use for target users?

• For each non-functional area are the following requirements specified?:
  
  • Acceptance criteria
  • Special cases e.g.) expected peak loads, volumes, times, limits/exception situations?

Summary
The use of requirements-content checklists can help prevent or identify the requirement issues and build confidence that requirement completeness and clarity is good. They also can be used as part of a continuous improvement process. If you encounter requirements issues, raise them as soon as possible, communicate the potential impact, and measure the actuals. Measures of the impact of requirements issues can be used to justify requirements improvements in later projects or phases.