CM: THE NEXT GENERATION - Evaluating and Selecting CM/ALM Tools

with your first couple of evaluations.  Instead, look at a couple of full ALM solutions first.  Get the big picture.  Then it's much easier to make the decision.  If you're afraid of the complexity, you won't know two things:  Am I covering the requirements adequately?  What complexities am I introducing by leaving out part of the solution?

If there are areas that you don't understand, bring a couple of others in from the team, especially those who are familiar with the broader application lifecycle.  Just get them to sit with you through a couple of 90 minute presentations and ask for their feedback.  Understand the big picture.  And vendors are a means of gaining that understanding. Ask them the hard questions, or if you're not comfortable doing so, put them down in your RFI.  Let others participate in forming the RFI so that you have an adequate range of questions.  

Your RFI is not going to get you answers to all of your questions.  What it will do is help you identify the vendors that are worth short-listing.  Most vendors will say yes to virtually every one of your requirements.  But a yes for one will mean "Click on Preferences and select..." while for another it will mean "We'll send two guys in and by the end of the week (and maybe the end of your budget) we'll have that set just right...".  So focus on the quantitative questions:  How many administrators should I have?  When will I hit a scalability focal point when I'll have to upgrade my hardware, etc.?  How long would it take to create a trigger to do ABC, or to add a menu button that does XYZ?  Get a feel for the complexity and level of administration.  Ask for reference sites and hit them with the same sort of questions.

Even better is to ask if you can install, load up, and evaluate the tool yourself.  Do you need training ahead of time?  How much?  Can I try it first without training?  

If you've not evaluated CM tools before, you're about to add to your expertise.  If you have already done so before, and are looking mostly for new players and new releases, you know how much expertise you will gain through this process.  It is always a technology learning experience.  But hopefully it will be more - a decision-making experience based on sound research and feedback from your team.

Money Talks
How confident is your supplier that theirs is the best CM/ALM solution?  They have invested part of their career into selling the product - but that could have been the only available job.  What about if we put 10% down and pay the rest in two more installments over the year - or walk away from it if we don't like it?  The supplier wants your business. If they believe in the solution, this shouldn't be a problem.  If it is, raise a red flag and suggest that you'll move on to the next solution.

Even if their solution is not the best, many vendors will let you jump through the hoops, using up your evaluation resources so that you don't have time to get to the better solutions.  So you really want to focus first on those tools that won't eat up your resources:

• There's no pre-requisite infrastructure to buy and install
• There's no up front tool cost
• There's not a large investment of your time in the evaluation
• There's minimal training and customization costs for the evaluation

Be reasonable.  If someone wants a few day's wages to come in