Validating Mission Critical Server Software for Reliability

Member Submitted

This paper talks about testing for reliability of server software, and highlights the challenges in planning, automation, execution, collection, and interpretation of the data for determining reliability as well as some practical challenges/difficulties faced. It also talks about the relationship of reliability testing to other areas of testing such as performance and functionality.

1. Introduction
In the current competitive Internet scenario, as the software products are expected to offer more and more complex functionality, expectations on higher reliability too goes hand-in-hand. Further, this is more critical in case of server software products. Needless to say, reliability of a server is a key factor both to the software developer and the customer, to make decisions, and hence a key factor for quality. The server software is expected to run without any failures for a continued period of time to support present business needs. In the e-commerce scenario where the server is expected to run round the clock catering to the customers throughout the world, spanning different time zones, there is no chance to reload or restart the server application. These requirements make the reliability testing more complex and critical.

This paper talks about "Software Reliability", the case of testing for reliability of server software, and highlight the challenges in planning, automation, execution, collection and interpretation of the data for determining reliability as well as some practical challenges/difficulties faced. It also talks about the relationship of reliability testing to other areas of testing such as performance and functionality.

2. Definition
Software reliability can be defined as the degree to which there is failure-free functioning of the software for a specified period of time or specified number of iterations/transactions, in a specific environment. Software reliability is an attribute and key factor in determining software quality.

Reliability of software describes the customer's expectation of satisfactory functioning of the software in terms that are meaningful to the customer. This description may be significantly different from the theoretical definition. The customer must tell the circumstances under which they will 'trust' the system that will be built.

The following are the key factors for reliability of the software:

  • No or very few errors from repeated transactions
  • Zero downtime
  • Optimum utilization of hardware resources
  • Less utilization of network resources
  • Consistent performance and response time of the product for the repeated transactions for a specified time duration

In a typical Client-Server scenario where the server products are assumed to be up and running all the time, it may be ensured performance of serving numerous clients without any considerable response delays by consuming optimum amount of resources of the system such as memory, CPU...etc.

The power and efficiency of the hardware resources have been increasing tremendously over a period of time and the software should be designed to utilize all the resources optimally to provide maximum returns for the investment of the customer. At the same time the server application under test, should yield these resources to other critical applications that may run on the same hardware as necessary.

In today's Internet scenario the server software performing reliably on a system is not enough, the implications of the software on the network are also equally important. The software should be designed to work utilizing less of network resources, as there are many customers who still have low network bandwidth and may be running other applications on the same network. So, a software reliability testing should also include the impact of the software on the network.

What is important is, the customer needs or expectations are described in a quantifiable manner using the customer's terminology.

Some examples of reliability are:
The system will be considered sufficiently reliable if:

  • 10 or fewer errors result from 1,00,000 repeated transactions
  • It supports X transactions per second throughout the specified period of time (i.e., there is no degradation of performance because of reliability)
  • The software optimizes hardware, software and net resources for repeated transactions
  • There are no side


About the author

TechWell Contributor's picture TechWell Contributor

The opinions and positions expressed within these guest posts are those of the author alone and do not represent those of the TechWell Community Sites. Guest authors represent that they have the right to distribute this content and that such content is not violating the legal rights of others. If you would like to contribute content to a TechWell Community Site, email

About the author

Srinivasan Desikan's picture Srinivasan Desikan

Srinivasan Desikan is the director of quality engineering at Siebel Systems, Inc., in Bangalore, India. Srinivasan has more than seventeen years experience in test automation, test management, test processes, and in setting up test teams from scratch. Srinivasan has delivered talks on testing at several international testing conferences, and delivers lectures and tutorials regularly at several universities and companies in India. Srinivasan has co-authored the book titled "Software Testing--Effective Methods, Tools and Techniques" published by Tata McGraw-Hill, India. He is currently co-authoring "Comprehensive guide to software Testing" to be published by Pearson Education in July 2005. Some of his interesting articles can be accessed at Srinivasan holds a post-graduate degree in computer applications (MCA) from Pondicherry Engineering College and a bachelor's degree in computer science from Bharathidasan University.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!