The Homegrown Tools Syndrome


Test management is a generic process, yet much effort goes into developing tools in house to do this work. Learn the reasons for this phenomenon and suggestions for avoiding it.

Test management is a rather generic process. The basic building blocks are the same: test cases, test suites, test cycles. How much different could it be from one team to another, or even among different companies? And yet, I have seen much effort spent developing tools in house to do this work.

Why does this happen? Why spend time and effort reinventing the wheel, when this wheel can be brought home from open source or commercial options?

And, it’s not just test management. I’ve witnessed this happening with other generic tools, such as test execution engines, bug tracking, and requirements management tools.

To me, the fact that the phenomenon happens across a wide range of tools is an indication that the reasons are predominantly non-technical. It’s not that test management is an art that is not yet fully understood by commercial tool developers and therefore has to be designed in house. There are other reasons at play that cause this seemingly redundant effort.

In this article, I propose a number of reasons that I think lead to internal tool development. While I refer to “test management tools” (TMTs), the logic applies to a number of other tool types.

"We Are Special"

The common wisdom when discussing tool selection is "Find a tool that supports your process." This is one of the main reasons why teams end up developing their own tools.

How's that?


User Comments

Kobi (Jacob) Halperin's picture
Kobi (Jacob) Halperin

Good points Michael,

Another issue is politics - choosing a full ALM requires consent of other group leaders (System Eng. and Dev), often not easy to acquire - so testers switch into temporary fallback, which result in large amount of duplicate work (rewrite) in separate tools + the loss of traceabilty and E2E management visibility.

Many are unaware of the vast offerings of free ALM tools these days (such as XStudio), and the ability to to adapt these tools by sending feedback and change requests.

Situation is even worst in automation frameworks - where many turn into bad coding habits, unique programming languages, and limited logging and mainly drill-down abilities.

Here the situation is worst, since most free tools are aimed at specific arena, and when one seeks to automate both legacy desktop GUI, web based GUI, Embedded CLI and test equipment - he must select one area and find bypasses for handling the rest.

Not to mention integrating between automation frameworks, ALMs, Bug Trackers, CMs and the dev IDE.


@halperinko - Kobi Halperin

August 28, 2012 - 1:31am
Oren Reshef's picture
Oren Reshef

I think you jumped to conclusions too soon in saying that "the fact that the phenomenon happens across a wide range of tools is an indication that the reasons are predominantly non-technical", I evaluated more than a dozen tools and couldn't find that one that will justify the effort in buying it, modify it, or migrate hundreds of test cases from out in house system.

In my industry of highly embedded devices most (all is a strong term to use here) TMTs requires a lot of customization, so you end up with a thin layer of test management that can be easily developed in house.

This also leads to another reason you missed- company size. In a company of more than 15,000 employees letting a small team (10-20 engineers including managers) develop such a tool is cheaper than buying it, and guarantees long term support.

August 28, 2012 - 4:41am

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.