This level of evaluation would include reviewing the output of the research evaluation that includes focusing on the top two or three SCM products. Then it includes defining clear SCM product requirements for the application’s needs. This would involve forming an SCM product evaluation team, gathering SCM product requirements, weighting the requirements by importance, getting a demonstration of each requirement from SCM product vendors (and asking how their product meet the requirements), and scoring each requirement. After the demos and scoring of the products, a selection may be made. Performing this level of SCM product evaluation, reduces the risk of incorrectly selecting the best tool to fit your needs since you have a good idea of your SCM product requirements and you have seen demonstrations of SCM vendor products, and hopefully determined at a high-level which SCM product best meets your needs.
This level of evaluation would include the research and demonstration evaluation. The top one or two products (or as many as desired) from the demonstration evaluation would be brought in-house for a 3-6 months to exercise and test the SCM vendor product(s) against the requirements specified and in a project scenario. The input from request for proposal (RfP) from each SCM vendor can be reviewed as part of the evaluation. The SCM RfP should ask about the how the vendor delivers releases and patches, how they solicit customer requirements, and how they price their product (amongst other things). Beyond this, you would want to establish an SCM pilot environment and select an application in which you would place under SCM version control. This environment should be similar to the working application environment (e.g., same platform, sample application code, development tools, etc). Tasks will include installing the SCM product, identifying an application (or pieces of code), and importing the code into the SCM product repository. This may entail establishing an SCM version control, build, and packaging process. Then exercise and test the SCM product(s) to determine if it meets the SCM requirements that were defined. This type of evaluation has the lowest level of risk associated with the selection of the SCM product.