Which is best path—fully integrated ALM or custom best of breed?

Which is best path—fully integrated ALM or custom best of breed?

Below are some observations I've made and I'd like to solicit the communities feedback on the conclusion I've drawn from it.

When looking at industrializing software development by creating a fully integrated and automated pipeline through which changes flow from check-in, through integration, build, testing, packaging and deployment; there are 2 paths one can follow:

1) Buy and implement a single vendor solution that covers the whole pipeline from check-in to deployment, the main drawback with this option being that most of the tools that make up the pipeline will do an OK job, not great, not the best in the industry but they will get the job done. Secondly there is the tie-in to a single vendor which is often perceived as a thorny issue because of the usual what-ifs and the constraints on customization of processes, tools etc.

2) Buy individual 'best of breed' tools to cover each of the stages in the pipeline and integrate these tools with your own resources. Creating this integrate pipeline with your own resources will take time, most likely a couple of man years to get a fully automated pipeline in place and then there is the support. Second because of the less than 'uber' tight integration the value of all the tools together is less than the sum of their individual value.

The above observations bring me to the conclusion that there isn't that much to be gained from buying the best of breed tools for the individual areas, sure you will have the most powerful tool for this and the other, but in the bigger picture that value won't be that noticeable. Just like a single vendor solution won't give you the best pipeline just because it's fully integrated as each of the stage which will be performed OK.

At the end of the day the value you will gain from both paths will be more or less equal, some areas will be better in one others will be better in the other.

What do you say? Did I miss something obvious?

- Antoni

Answer Vote

I'm with Joe the Wincer on this-- when a vendor claims they have an integration, it means nothing to me at all.

First, you have decide *why* do you need the integration. Mostly the vendor's requirement is to claim they have an integration so people will buy their product. Lots of naive people will buy only on that basis, and not validate that what the vendor's product does is what they need done in their organization.

I lean towards a best-in-breed preference so long as tools in question have an API I can use to access any functionality I need. Then I know minimally that I have the building blocks I need for whatever solution a given client has requirements for.

Somebody has already mentioned the cost of maintaining that custom "glue", which I acknowledge is a factor. But then, so is the cost of buying a tool that turns out not to meet your needs. Or the cost of discovering that you thought you bought an end-to-end solution only to discover that now there is a new technology you want to use that doesn't fit in your chosen solution. Or the cost of discovering that your IT assets depend on your chosen vendor's good graces, because you can't swap it out for an alternative.

To mitigate the cost of glue, an organization could subject the glueworks to the same code review and testing rigors they use for their other IT IP.

--Bill

Answer Vote

That is a fairly complex question to ponder. I usually say that I prefer the best of bread approach. Of course I would hope the that best of bread tools provided good APIs/scripting language support (python/perl/etc) so that they could be integrated with other tools.

I suppose the plus side of using tools from one vendor is that you save time on coming up with all that integration.

One big downside of using a large suite of tools from one vendor is the possibility the vendor support because bad or the vendor goes out of business. If all your tools come from that vendor you have to replace them all over time. If they come from different vendors you only need to replace the one.

I would think that both can work well and there is no right answer.

Answer Vote

I would like to take this just a bit further. When we talk about a single integrated solution by a single vendor, there are 2 categories:

1) Those solutions which integrate separate tools, possibly best-of-breed tools with others, but all supported by a single vendor who did and supports the integration.

2) Those solutions which create an integrated ALM suite on top of a common core - common database, administration, user interface, customization capabilities, process engine, etc.

There is a huge difference between these two. MKS goes a fair ways toward the latter, but not all the way. Neuma, another Canadian company (where I work), goes all the way - and even lets the customer create additional "tools" using this engine.

For those who have seen CM+, it's clear that there's no contest. An ALM suite on a decent common core has so many benefits. Like multiple site working across all tools. Like consistent backups. Less training. Rapid traceability navigation.

The best part is that changes made to the core benefit all of the tools. And typically, as in Neuma's case, the core contains various high-level customization capabilities.

I've seen several "ALM" backplane schemes launched, only to fail, or else only to serve the initiating body.

The best case would be if the integrated toolset based on a common core would be the best-of-breed of each tool. Does Neuma's toolset meet this? That's not for me to judge. But by continuing to focus on the core capabilities, and in particular the ability to rapidly and extensively customize (e.g. build custom dashboards in a few minutes, add new "tools" in a few days or less) a customer is in more control of the end product.

And giving the customer control of the end-product goes a long way to defining "best-of-breed" because everybody's definition of "best" will differ.

Answer Vote

Antoni,

What I have discovered is that depending on the organization and its size a person may need access to two or three of the many parts of ALM. Most of those want a one stop shop. The fact that they have to move from application to application is seen as a negative. I wouldn't mind having one tool and having a mechanism to turn off the pieces that are not needed through security. That would be nice, however, some of the ALM offerings are the result of companies buying different tools providing a link from one to another and saying they are integrated, when in fact, the industry hasn't even defined what integrated means yet. ALM is still in its infancy, or at least the direction of ALM is in its infancy.

My two cents,

Joe

Answer Vote

interesting observation on what integrated means, and you are absolutely right in most cases integration only goes so far as to give you a link to another tool.

Personally when talking about integrated solution its the management of / reporting from the system that I find more important, rather than whether all operations can be done from a single UI because in most cases its going to be different people doing the work.

- Antoni

Answer Vote

baynes and tekkie,

I agree with both of you, I would like to take this a little deeper. I think the keyword that vendors misuse integration. A link from one tool to another is not integration, or at least not tight integration. So anytime I hear the word tools are integrated I wince. Another popular name for faux integration is plug ins. Vendors will say hey our tool has a plugin to make it work with our RM tool or Version Management tool or our Incident Management tool.

Integration to me is you can operate in the integrated tool and not have to go back and forth. So in my RM tool I can check in and check out code as well in my VM tool without having to leave the RM tool. So I say be careful when people say a tool is integrated.

Regards,

Joe

Answer Vote

You are right that you need to consider the cost of integration. However many best of breed tools come with integrations to other best of breed tools - so you don't always have to do it yourself. Point tools are particually open to integration and tend to have many existing integrations. Whereas many comprehensive ALM systems tend to be more closed so if you need to integrate them to something else (eg for legacy or because it is outside their scope) it can be more difficult than with a best of breed point tool. Some of the web based ALM/information sharing systems (eg CollabNet) are themselves integrations of best of breed point tools with some propriatory code.

7 Answers

Answer Vote

baynes and tekkie,

Regards,

Joe

Answer Vote

interesting observation on what integrated means, and you are absolutely right in most cases integration only goes so far as to give you a link to another tool.

- Antoni

Answer Vote

Antoni,

My two cents,

Joe

Answer Vote

I would like to take this just a bit further. When we talk about a single integrated solution by a single vendor, there are 2 categories:

1) Those solutions which integrate separate tools, possibly best-of-breed tools with others, but all supported by a single vendor who did and supports the integration.

2) Those solutions which create an integrated ALM suite on top of a common core - common database, administration, user interface, customization capabilities, process engine, etc.

The best part is that changes made to the core benefit all of the tools. And typically, as in Neuma's case, the core contains various high-level customization capabilities.

I've seen several "ALM" backplane schemes launched, only to fail, or else only to serve the initiating body.

And giving the customer control of the end-product goes a long way to defining "best-of-breed" because everybody's definition of "best" will differ.

Answer Vote

I suppose the plus side of using tools from one vendor is that you save time on coming up with all that integration.

I would think that both can work well and there is no right answer.

Answer Vote

I'm with Joe the Wincer on this-- when a vendor claims they have an integration, it means nothing to me at all.

To mitigate the cost of glue, an organization could subject the glueworks to the same code review and testing rigors they use for their other IT IP.

--Bill