One of the main attractions of agile methods over traditional, heavyweight approaches to software engineering is its ability to accelerate the software development process. By minimizing superfluous activities and artifacts, such as models and documentation, and focusing instead on coding, agile methods can increase productivity and reduce overall development time.

Focusing on coding, though, also has a down side. It means that new applications are typically written entirely from scratch. Software reuse, as envisaged by McIlroy back in 1969 [1] at the conference that coined the terms "software engineering" and "software crisis", is not explicitly addressed in the current generation of agile methods. If it takes place at all, reuse tends to be done in an ad hoc, unsystematic way.

Until recently this was not a major issue, as fine grained component reuse was not a cost effective proposition.  There simply were not enough good components around and the technologies available for finding them were too inaccurate. As a result, systematic attempts to reuse software did not pay off because the likelihood of finding suitable components was too low when compared to the effort required to find and evaluate them. However, this situation has changed over the last few years with the dramatic increase in the amount of open source software freely available over the Internet.  The emergence of high-performance code search engines such as Koders, Google Code Search and Merobase dedicated to indexing and support searches over these code resources has also contributed to its popularity. For the first time, these technologies make fine-grained component reuse a viable proposition, by reducing the effort involved in finding and reusing components to a similar or lower level, when compared with the effort of building them from scratch.

All kinds of projects stand to benefit from this enhanced opportunity for reuse, but agile projects may benefit the most, as they specialize in development at the level of fine grained components. They also  offer the same basic value proposition as software reuse: accelerated development. Additionally, they emphasize the description of what components should do (i.e., tests) before the description of how they should do it (i.e., implementation) and a ready-made opportunity to search for potential reusable components before they are implemented from scratch. In short, there is a high potential synergy between agile development and software reuse; this can further accelerate the software development process and, ultimately, lower costs.

In this paper we introduce a software tool developed by the Software Engineering Group at the University of Mannheim that aims to promote this synergy. Code Conjurer [2] is an Eclipse plugin that is driven by the Merobase component search engine. The tool is released as open source under the GNU/GPL license.

Proactive Reuse Recommendation
The basic service, offered by Code Conjurer, is used to search for reusable components (e.g., Java classes) based on a description of an interface or API (i.e., set of operations). This description can take various forms, such as a Java code module (e.g., interface or class) or even a UML class. Suppose, for example, that a developer has decided that he wants a matrix supporting a certain set of operations.

Once he has defined a part of the interface that is supported by the desired component, Code Conjurer can find suitable reusable components that already implement this interface by using the Merobase search engine. In Figure 1, which shows a screenshot of an Eclipse environment including Code Conjurer, the user has defined parts of the required Matrix as a UML class (top right hand window). After sending a search request to Merobase, Code Conjurer lists the discovered components in the reuse recommendation window (lower central part of Figure 1). It first displays the results and then fetches the source code so the developer can instantly browse through the interesting reuse candidates (lower right part of Figure 1) and access relevant metrics such as LOC, Halstead and complexity measures, etc.

If the developer believes that a component will be useful, he can insert it into his workspace with a double click of his mouse. Code Conjurer has an automatic dependency resolution feature that can be employed in the background to find any other classes that the chosen component depends on. After analyzing the component, the tool can trigger new searches that can find missing components that can then be inserted alongside the original. In many cases, all dependencies can be resolved and fully functioning, executable code can automatically be added to the development project.

Code Conjurer can also be set to "proactive mod," in which it monitors the work of the developer in the background and triggers new searches automatically every time the interface-defining part of the component under development changes (i.e., whenever a method is added, changed or removed). The developer can, therefore, proceed with his normal work, undisturbed, until he decides that one or more of the recommended reuse candidates are worthy of attention. He can also trigger a manual search for reuse recommendations or use Code Conjurer's quick fix integration feature that searches for reusable assets when a type in the developed Java code cannot be resolved.

Test-Driven Reuse

This proactive recommendation capability can be valuable in many kinds of development projects, including traditional heavyweight projects. It does not, however, exploit all of the opportunities that exist to accelerate software development. In particular, it does not exploit the fact that discovered software components are potentially executable and that test-cases are often written before the implementation code, especially in agile projects. Code Conjurer offers another more advanced search facility, known as test-driven search that leverages these two facts to significantly boost the value of search results. As soon as an executable test has been defined by the developer, Code Conjurer sends this to the Merobase server to find matching components that pass the test. Merobase does this by:

• Working out what interface the test is written against
• Performing a normal search on this interface
• Testing the results to filter out those that match

The resulting set of component recommendations is of much higher value than that generated by regular interface-based matching alone because the components do what the developer has specified; that is, they pass his test.

A developer can decide to write the whole test before initiating a test driven search (as shown in Figure 2) or configure Code Conjurer to search for components within the test/implementation cycle advocated in Extreme Programming [3]. In both cases, Code Conjurer indicates how the search is proceeding (bottom left corner and, optionally, in the other two windows at the bottom). Whenever a candidate component passes the given test, it is highlighted in green using the familiar JUnit convention. If the test is complete, a component that has passed the tests (indicated in green) is fit for purpose and can be inserted directly into the project. If not, the developer can complete the test using the candidate component's interface to check if it behaves as required.

When performing test-driven searches in proactive mode, Code Conjurer is able to suggest candidate reusable components based on partial tests without disturbing the developers normal work and before much of the component has been implemented. It complements Extreme Programming by promoting the reuse of component implementations alongside the traditional approach of building them from scratch. We have, therefore, referred to this approach as Extreme Harvesting in previous papers [3]. Extreme Harvesting extends the extreme programming mantra of "design a little, code a little, test a little" to include "design a little, harvest (i.e. reuse) a little, test a little" as a possible route to successful application development.

Using Code Conjurer it is not only a venue for locating normal reusable components based on tests defined by the developer, it also possible to search for pre-existing tests, as well. When a developer begins writing a test, Code Conjurer can look for previously indexed tests and offer these for reuse.

Conclusion
When Code Conjurer is able to find components that match a developer's tests, it effectively "conjures up" reusable code from "thin air". The user does not have to deviate from his normal development work until a component has been found that does what he is trying to build. Even when no components are found that pass a developer's tests Code Conjurer is able to provide valuable information. By analyzing the search results using various clustering techniques it can create a characteristic picture of the group that includes the typical set of methods. For example, when searching for a matrix component based on an interface with just a multiply method, Code Conjurer can indicate that components with this method typically also support 13 other methods as well, such as swapRows(), transpose() or findPivot(int).

By unobtrusively adding test-driven search capabilities to mainstream development platforms such as Eclipse, Code Conjurer is able to genuinely speed up software development projects, whatever methodology is used. However, agile projects that specialize on small-grained components and writing test cases upfront, stand to gain the most from the tool. Code Conjurer is one small step for test-driven development, one giant leap for agile projects.

References

1)  M.D. McIlroy. "Mass produced software components," Software Engineering Concepts and Techniques, pages 8898, 1969.

2) O. Hummel, W. Janjic and C. Atkinson, "Code Conjurer - Pulling Software out of a Hat," IEEE Software, August/September, 2008.

3)  O. Hummel and C. Atkinson: "Supporting Agile Reuse through Extreme Harvesting," International Conference on Agile Processes in Software Engineering and Extreme Programming (XP2007), 2007.

About the Authors
Oliver Hummel currently works as a consultant for Perot Systems Germany in Frankfurt. Prior to that he used to work for the Chair of Software Engineering at the University of Mannheim where he received his PhD degree with a dissertation on software component retrieval. Oliver's research interests include software reuse, information retrieval and software development processes. 

Werner Janjic is a research student at the chair for Software Engineering at the University of Mannheim. Werner’s main research interests are practical software reuse in the context of agile development and its impact on the software development lifecycle. He received a diploma in business and computer science from the University of Mannheim. Contact him at the Institute for Computer Science, University of Mannheim, A 5, 6, B238, 68131 Mannheim.