CM: THE NEXT GENERATION - What Else Does ALM Need?

for which you have figures, scale your actual figures (i.e. divide by your number of users and multiply by 100).  If you don't keep these metrics, take a few minutes to think about outages.  Did they affect one person (1 hour/hr) or everyone (100 hours/hr)?

Scoring: Number of User Hours Down per 100 Users per Year

1. >10K  
2. <10K  
3. <5K   
4. <2K  
5. <500

3.  Global Operation at Multiple Sites
Does your solution support multiple sites.  Now we're not talking about a single function of your solution.  We're talking about your whole solution.  It might be OK for some Requirements Management functions used only by one person not to be supported across multiple sites.  But it's not OK for CM, Testing, Developer, etc. functions.  Ask:  Can I go to another site and still perform my job?

Scoring:
 

1. Single Site Operation
 
2. Multiple Site Thru Partition / Resynch Operation
 
3. Central Site Operation with Web/Remote Client
 
4. Multiple Site Full Operation from any Site
 
5. Full Multiple Site + Can Keep Data From Select Sites

4.  Low Level of Operational Administration
I've seen some great tools that are simply too expensive to operate.  They preclude usage by the majority of smaller shops.  How does yours measure up.  You may need someone trained as a backup person, but if that's all they are, you shouldn't count them as a separate administrator.  If you have 10 people spending 25% of their time each on CM/ALM operations that keep the solution running, that's 2.5 administrators.

Scoring:


1. >2 Administrators per 100 users
 
2. <= 2 Administrators per 100 users
 
3. <= 1 Administrator per 100 users
 
4. <= 0.5 Administrators per 100 users
 
5. <= 0.1 Administrators per 100 users

5.  Rapid Installation/Upgrade and Data Population
(Based on 10,000 records, 2,500 files x 2 revisions)

OK, the solution may be running now, but if you had to start from scratch, how long would it take you put the solution in place, assuming you have good knowledge of how to use the solution.  Look at times it takes to do installation (some tools are just a few minutes).  If you have to install on multiple machines, multiple the average install time by the number of machines.  What about an upgrade to a new major release - how long?  Again, muliply if you have multiple machines to update.  And what about initial data population.  We're talking about source code, documents, problem reports, etc.  But to be fair, normalize your result based on 10,000 records and 2,500 files with an average of 2 revisions of each.

Scoring:
 

1. Install, Upgrade, Data Population each > 1 day
 
2. Install, Upgrade, Data Population any one < 1 day
 
3. Install, Upgrade, Data Population all in < 1 day
 
4. Install, Upgrade, Data Population all in < 3 hours
 
5. Install, Upgrade, Data Population all in < 1 hour

6.  Level of Integration Across Functions
It's great to have multiple functions supported in your ALM tool.  But if each function (apart from the actual process) has to be learned separately, that a big learning curve.  If each has to be supported separately on separate infrastructure, more learning curve.  Not to mention more difficulty for use.  How cleanly integrated are your ALM functions.  Look primarily at end-users, but also at administrators, CM staff and other users.  By Multiple User Interfaces, we don't mean separate role-based interfaces - we mean - is the look and feel different across functions, or do I have to exit one interface and enter another to perform a role.  Either of those constitute multiple interfaces.  For Databases/Repositories, your source code, documentation, problem records, baselines, etc. all have to be considered.  If you have separate