A tester testing real-time enterprise applications requires technical know-how and expertise to be able to think through scenarios in which the application under test could behave abnormally in production. Beyond the functional, exploratory, boundary, regression and performance testing, a quality engineer has to wear a software architect's hat and start thinking like one. He needs to consider the hardware and software modules on which the application will be deployed in production and ensure application availability when a module or a combination of modules partially or completely fails.
On the negative scenarios side are tests to check that there is no client data loss when the server side has a failover, how ctrl+N behavior (the keys to spawn new browser instances) has been implemented when simultaneous logins to the application are disallowed, and how user notifications are handled when required plug-ins such as applets and flash are not installed.
On the server side, it is always important to ensure that the pre-production, staging, or user-acceptance testing environment is as close as possible to the production environment in terms of network architecture and ports hardening. Often, an application that works in QA might fail in production, as it may not have been tested for port availability.
During load testing, the tester should use the cloud to simulate external load, as using internal servers does not simulate actual user bandwidth usage. Also, for applications with load balancers that serve multiple servers, more than one cloud machine should be used to distribute the load and avoid all requests going to just one server. Component failover at all levels is a very realistic scenario, and test cases should be defined to test all component levels—web server, application server, and database server.
For testing messaging applications that are processing or publishing live data, testing should be done by adding a “slow” consumer or subscriber to avoid system-health deterioration caused by messages that are in memory for a long time. Herein, tests should also check for proper system notifications and alarms. Also, use a full data set during load testing to expose potential issues related to simultaneous data actions. Such issues may not surface with partial data sets
A quality engineer thinking like an architect can unearth potential production issues by focusing on the architectural components that the application will run on. Resources are a constraint, especially in real time applications, so it is important that the test plan contains design tests to cover such cases.
To promote efficient usage of resources on the web-server side, the tester should ensure that static content is rendered from the web server instead of the application server. Similarly, system compression levels should be adequately tested, as bandwidth usage is often a concern in real-time applications.
On the application-server side, the tester should try things including forcefully shutting down all parts of the application, like the admin server and SNMP server, to verify that the application is responding as expected. Verifying that that components reconnect and communicate with each other when they are restarted is also important. To verify real-time server clustering, tests should cover test-environment hardening and check for port usage in production. While testing for clustering, the tester should use a similar number of nodes as in production to uncover any performance degradation as the number of nodes increases. Finally, testing the server-side cache’s performance should be done with a full set of data to help tune the cache effectively, since responses vary heavily as data sets increase.