As we start a new year, the CM/ALM world continues to evolve. We've seen a lot of migration to open source solutions, but we've also seen, in the past year, some impressive commercial ALM releases, especially from IBM (RTC) and from Neuma (CM+7). It appears clear that those who can do with less will move from lower end commercial solutions to open source. Those who need more comprehensive and integrated solutions, targeted at the wider project team will continue to seek them commercially.
I thought it might be fun to start off this column for 2011 with a look, not at the CM/ALM functionality itself, but at what else ALM needs - the underpinnings and structures that are in place within a CM/ALM tool, which help give the tool significant character. Rather than pointing at any one tool, or even characterizing tools as a whole, I'll present this month's contribution in the form or ALM tool requirements - that is, requirements for adopting a CM/ALM tool that fall outside of the specifics of the CM/ALM functions themselves.
So for example, rather than focusing on the the ability to automatically generate baselines, do fancy merges or create an elaborate branching scheme, we'll focus more on things such as ease-of-use, traceability navigation, and process support.
So that you may use these requirements to evaluate your current tools in these non-functional areas, I've even included a way for you to evaluate your current solutions. First, define what you see as your solution - and then look at the criteria to see how you see your solution fitting the criteria. I'd be interested in hearing results. Just post the solution description and your score at the end of the article. Hopefully we'll get a representative sample. After we get a few postings, perhaps I'll share my own environment's score.
In my opinion, it is in these areas, more than in the CM/ALM functional requirements, that we'll see the biggest improvements over the coming year(s). Yes they are functional areas of the tool, but not requirements of each ALM function per se. If you're looking at a prosepective tool, look closely at these areas. We'd love to see your results and I'll gladly publicize a spreadsheet of results if I hear from at least 3 of you. I'll post it against this article.
The scoring criteria are my own. You may not agree with them and your comments are more than welcome. They are in the absence of any weighting criteria, so each is significant on its own merit. If you have some other criteria you think might be added to the list, let's hear from you too.
The evaluation is heavily geared toward ALM, rather than strict CM or SCC systems. All criteria are evaluated on a scale of 1 to 5. Pick the closest one or use .5 modifiers if you wish. In some cases, the criteria are clear. In others, there are a range of factors to consider. In this case, an estimate of the percentage of factors your solution excells at will do. However, results will vary because "excells" is subjective.
1. End-to-end Functionality Coverage
OK. We're not looking at ALM functionality in detail here - just what areas of ALM your solution covers (i.e. sufficiently that you don't need additional tools to support that function.
Scoring: Start with 5 and subtract 1 for every 3 functions missing)
- Multiple Product Management
- User Management
- Requirements Management
- Document Management
- Problem/Defect Management
- Feature/Activity/Project Management
- Version Control/Baseline Management
- Change Package Management
- Build/Iteration Management
- Test Case Management
- Test Suite Management
- Release Management
- Deployment/Site Management
- Customer Request Tracking
There are many ways to evaluate reliability. How many modes of redundancy. Disaster recovery capabilities. Mean Time Between Server restarts. And so on. But we're going to put these aside and bring it down to nuts and bolts.
The bottom line here is, in a 100 user environment, how many user hours are lost per year due to non-availability of the solution. It might be because of upgrade, problems, or even network outages. If you have a smaller/larger environment