CM: THE NEXT GENERATION - Evaluating and Selecting CM/ALM Tools

and understand your process and customize the tool to that process prior to your detailed evaluation, that's good money spent.  If someone wants to put one or two people on your staff for the course of the evaluation, that's fine - at their cost - not yours.  If someone wants to run the evaluation for you - red flag.  They may be trying to help you, or they may be trying to hide complexity from you. Identify which before proceeding.

If the contract is non-trivial, a vendor should be willing to invest a few hours to a few days pro bono.  Rather than taking a customization course, have them walk you through a customization effort that they can perform for you.

The bottom line is that as long as you hold on to your money, they have to sell you the solution.  And with such a backbone technology, you really shouldn't let go of the money until your team is up and using the tool.  Start with a pilot if necessary.  But make sure you know what you're paying for and what the total cost is.   I've seen too many companies spend all of their money on the "best" solution, only to find that it was a money sink which mostly meets today's needs, but won't grow into tomorrow's.

Wrapping Up
Evaluation is a bit more complex than you may have thought.  But put the effort up front, so that you understand what you need. And don't commit your resources until you're sure it is a working solution in your environment. If you hit a tool that's expensive and/or time-consuming to evaluate, set it aside and go on to an easier one, so that you don't chew up your resources on the difficult ones - they're probably hard to use and costly to administer anyway. And you can always come back to them if you don't find something better.

Finally, if you're looking for more detailed evaluation criteria, sorted by CM generations, checkout the articles in the November 2008 CM Journal.