In today's IT environment, service-oriented architecture (SOA) adoption continues to be an important strategic approach for next-generation initiatives. SOA testing is "on the rise," according to Gartner's Hype Cycles 2011 research report. A major driving trend is the effort to migrate from legacy platforms such as client-server to a more robust, platform-agnostic SOA that is flexible enough to serve the needs of multiple corporate business objectives through a shared technology platform (web services). While this trend has been underway for several years, the Gartner study suggests that many organizations are still actively making the switch.

SOA is a business enabler, and the business—not IT—drives SOA. SOA supports active business stakeholder engagement throughout the project and offers an opportunity for you as a testing professional to foster active customer participation to deliver the highest quality product possible. Your job is to develop a comprehensive strategy that leverages SOA-specific testing practices to shape a holistic testing approach.

Challenges Ahead
There are some clear challenges for testing organizations transitioning to SOA, such as determining which legacy testing practices will transition well and which will not, preparing the organization for the necessary phases of SOA testing, and preparing for SOA-specific test tools and other needed resources. A very important aspect of your job is to ensure that you have the necessary information. While many of the testing practices used for legacy testing efforts are still relevant and necessary, the organization must adopt new testing techniques and approaches and change its mindset in a few key areas of the software development and testing lifecycle (SDLC).

Know the Important Aspects of a Comprehensive SOA Testing Strategy
Today's SOA components demand more from testers than just business-level understanding of requirements. This is because an effective testing strategy often will require early involvement to have the best possible chance of creating a high-quality software product in a reasonable amount of time. While SOA is made up of much more than just web services, they are a major part of a SOA testing strategy and, as such, a general understanding of them is critical.

Because SOA components are often made up of atomic web services that are heterogeneous and configurable to work with other web services, the testing approach must take into account the highly variable nature of service independence. The large number of permutations that can arise from several services that must interoperate in different ways is very similar to what occurs on certain API-level client-server testing projects. A testing strategy must allow for non-GUI testing approaches that prove components interoperate per a defined services (or API) specification. This type of testing is typically referred to as headless testing. Because services are developed independent of and typically well before a GUI, the need to test a service early in the SDLC is critical. One main difference, though, is that the types of tools that you use on an SOA project will certainly be different than those you use for API-level client-server testing. The tools are not necessarily more complex, but using them to their full potential requires a level of architectural understanding. There are a number of commercial and open source web-service-specific tools that can help with the early testing phases of services.

Know the Most Important Components of Your Organization's SOA Architecture
While there will be many components in your organization’s architectural model, you should focus on the components that are going to help you serve the critical objectives defined in the testing strategy. Toward that end, there are at least three important architectural elements you should know.

The first element is the Web Services Description Language (WSDL). Having a general understanding of the WSDL file elements is important, particularly as many testing tools in the marketplace will utilize the WSDL file as a starting point. WSDL files describe a given service to consumers by specifying the message type, port, supported operations, data types, and all other attributes of how a service works. One of the main benefits of a published WSDL is that it indicates the structure of the XML messages that a web service can process. At its most fundamental level, the WSDL defines the contract between the producer and the consumer of a web service. This is very important information for a test professional to have during the planning process.

The second element is how services interoperate in your particular architecture, or what I refer to as the services control flow. Interoperability testing is a fundamental component of an SOA test strategy and will paint a clear picture of how services relate to and interact with each other. It also will help you understand the data flow involved. With a clear understanding of the control flow, you can plan individual test cases for all of the service components in your project, including third-party and external partner services.

The third element is the service-level characteristics of the web-service implementation that will help you determine the proper testing approach with the tools you have available. How the web services under test for your particular project have been implemented is crucial to your understanding of the testing task at hand and your subsequent creation of an effective testing strategy. This implementation includes the service-level characteristics and any run-time behavior that has been defined in your environment. This type of testing activity often falls under SOA-governance testing, which covers interoperability verification of services to ensure that internally and externally consumed services follow an approved design standard. As with other software development activities, there are several ways to build web services that utilize different programmatic mechanisms. Web services that have been developed in Java—for example, using the Java API for XML Web Services—have different characteristics than those developed in .NET. This could affect the way that a given testing tool interrogates a web service, so you need to be aware of the specific service-level characteristics.

Develop an End-to-end SOA Testing Strategy
As cliché as it may sound, "test early and often" is one of the most critical elements of an enterprise SOA testing strategy due to the way that SOA components are developed, maintained, and supported. This suggests a thorough, end-to-end approach to testing SOA, because testing involvement begins during the early architectural phases. Typically, a solution architecture team defines technical service characteristics at a macro level from a set of business requirements. Once the macro-level business characteristics are defined, the architecture team then creates the micro design, which includes specific web services to satisfy the business requirements. The next step is for the development team to implement that micro design by coding web services following an architectural specification—likely adhering to a defined reference architecture that provides guidelines for organization-wide web service development standards. There are various opportunities throughout the architecture and development phases to introduce quality checks, so testing early and often should be a recurring theme in any SOA testing strategy.

For the earlier architectural phases that include the macro design and the micro design, this is an opportunity for a tester to become familiar with the initial business requirements and how they map to the service-level characteristics. From there, understanding the next phase, web service design, is critical. This is where the development team will use the reference architecture to code the needed services. For a testing professional, having a general understanding of the reference architecture used to code the web services can provide a great deal of information. For example, you will learn implementation attributes about the services that you will be testing, which will be a critical input for both test-planning activities and test tool configuration.

Ensure SOA Performance, Security, Environment, and Tool Readiness
Some other critically important testing activities include both performance testing and security testing of web services. There are a number of open source and commercial tools available for performance testing. A main concern with performance-testing services is that you must ensure that your simulated workload models account for the interoperability of your service model under test, including the various data models that you need to test. For example, a web service may access a particular data repository and model to satisfy one of its service components, but the very same web service could then access a completely different data store for another component of its functionality. You should be aware of variations in both the service control flow and the data that it uses so that you can properly plan load-testing workload models.

Security testing has become an important part of SOA and web services certification. This is due to the reusable nature of services that pass sensitive data across corporate infrastructures and ultimately to external organizations and end-users. A comprehensive certification must include testing efforts that ensure the data remains safe and is not vulnerable to hackers. Security testing can be a difficult challenge due to the number of vulnerabilities that may need to be checked in a given web-services implementation. This effort can be partly automated with various scanning tools that are designed to interrogate web services for vulnerabilities. The Open Web Application Security Project is a nonprofit organization that provides some very valuable resources for testing web-service security. It offers standards for testing some of the most common vulnerabilities that web services expose, such as XML structural testing, WSDL testing, and testing SOAP attachments. In addition, there are concerns about data-related trust, integrity, privacy, and access control that will need to be tested. Because security testing covers such a broad spectrum, it is best to bring in specialists who can help define a comprehensive security-testing strategy.

The testing environments needed for SOA must simulate the production environment dependencies, including all of the downstream and upstream connectivity points that web services communicate with across the infrastructure. During the development phase, a common challenge that testing professionals encounter is limited testing environment connectivity. One way of addressing this is to write stubs that can simulate the dependent systems needed to communicate with the web services under test. Some of the tools available for web-services testing support this concept of stub creation. Another way to address this is to use one of the SOA virtualization tools that have been created specifically to virtualize an SOA environment so that testing can take place very early in the SDLC. There are benefits and drawbacks to both methods, so be sure to research them thoroughly for your particular environment.

Conclusion
Ultimately, your effectiveness at developing a holistic testing strategy for SOA and web services will be determined by how successfully you have planned for and adopted many of the testing considerations covered in this article. Given the importance placed on SOA to serve the needs of so many business-driven IT projects, your ability to use this information to ensure the highest possible quality products is crucial.