Migrating Data Successfully


database to achieve the correct results.

Using a test database serves two purposes: The development team gets a real picture of what is expected of the conversion routine, and the operational system is not subjected to any conversion until the data migration utility is stable and yields expected results. This ensures that we do not corrupt or create inconsistent data with the real system data.

Use Iterative Development
A conversion utility is designed to cater to a particular customization and implementation project. It is difficult to get the complete conversion working correctly on the first attempt. When data is converted using a standard utility, often data records will be converted incorrectly or only partially. Listing all the possible problems in data records of the existing system is a difficult task. The best way to deal with the missing pieces or defects is to use an iterative approach. Once the conversion utility is developed, it can be executed on the test database.

Closely examine the resultant data, then and then run critical business processes on this data. The defects found should be due to missing data, incorrectly converted data, or duplicate data. Resolve your problems and repeat the conversion. After a few iterations, a stable and relatively bug-free utility will be developed. The number of iterations will depend on the nature of defects and inconsistencies found after the test migrations. The extent and severity of the defects will depend not only on the size and complexity of the converted data but also on the quality of the data that is subject to conversion.

Have a Comprehensive Test Plan
Like any other application development, it is important to have a comprehensive test plan to achieve the required level of quality for a data migration tool. The only major point to keep in mind is that the test plan used for testing a conversion routine should focus on data integrity, apart from other test standards. There should be adequate room for regression testing that stresses critical business processes. These regression tests should be executed repeatedly at the end of each development iteration and after bug fixes.

The test plan should accept new and converted data while executing the critical business processes to ensure that users can work with both types of data after the data migration is completed. Use of a wide range of data will uncover many errors and help build a robust and effective conversion utility. Using a copy of the actual working database will give the test and development teams a truer picture of problems and their solutions.


About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.