Data Completeness
Data completeness validates that all expected data loads are integrated into the data mart, data store, or warehouse. This includes validating that all records, fields, and the full contents of each field are loaded, including:
- Record counts between source data and data loaded to the receiving database repository
- Tracking and writing out to file rejected records to a suspense file
- Unique values of key fields between source data and data loaded to the repository
- Populating the entire contents of each field to validate that no truncation occurs at any step in the process
- Testing the boundaries of each field to find any database limitations
- Testing blank and invalid data
- Data profiling (range and distribution of values) between source and target
- Parent-to-child relationships (part of referential integrity)
- Handling and processing orphaned records
The Business Data Layer
Two examples of BI tools are the semantic layer and the OLAP cube. A semantic layer (developed by the company Business Objects) is defined by Wikipedia as "a business representation of corporate data that helps end-users access data autonomously using common business terms." [2] The key point is that it creates a consolidated view of the data across the enterprise, in which all the data elements, table joining, relationships, and other configurations are defined.
About the author
Paul Fratellone is program director of quality and test consulting in the testing business unit of MindTree. Paul’s career of more than twenty-five years in information technology has been concentrated in testing, compliance, and quality management. He strives to achieve consistent execution to attain a predictable level of quality that is commensurate with the investment and enables leadership to objectively measure the success and continuous benefits from these investments.
