Driving Efficiency and Effectiveness with Web Analytics and Risk-Based Testing

[article]
Summary:
Web analytics can help you deduce, reduce, and prioritize your testing efforts. Learn how to gather and use qualitative and quantitative information about your users and the risks that can threaten your software's success.

One of the clearest definitions of web analytics is “the analysis of qualitative and quantitative data from your website and the competition or industry.” The data helps drive continuous improvements for the end-user experience, and what is being measured—website interaction (e.g., search, form completion, loyalty programs, download registration), social media interaction, in-bound and out-bound advertising, and performance—varies as much as the available tools. In the end, it’s all about sales revenue, transaction volume, customer stickiness, page views, length of stay, and advertising revenue.

According to Wikipedia, risk-based testing (RBT) is “a type of software testing that prioritizes the tests of features and functions based on the risk of their failure—a function of their importance and likelihood or impact of failure.” A question often asked of test owners is “What business risks are being mitigated and measured with RBT in place?” A good QA team can qualify the risk, but a great QA team can quantify it. If you are not measuring, then you are not managing, and if you are not managing, then you are not in control of your risks. Validate that your company has an overall understanding of risk identification and management before you introduce RBT.

Web-analytics tools try to provide insight into the success of marketing and sales campaigns as well as new functionality or feature sets, so it is most common for sales or marketing to fund and own them. If they don’t own them, then the product owner usually does.

Testing teams can use web-analytics statistics to quantify the need for both increased depth of testing and robust test suites (functional and regression) that mimic end-user patterns and behaviors, based upon a prioritized view of risk for insufficient testing. They measure the risk of untested or minimally tested code in the production environment, and they can directly tie the number of tests to business risk when the time it takes to adequately test the application is less than the time available.

Clarifying business requirements and goals is critical to identifying the correct analytical tool. From a test perspective, there should be few surprises about business requirements, criticality, and goals, as all tests should take these elements into consideration.

Web Analytics 101

Most users have encountered the omnipresent hit or visitor counters on websites. This is web analytics in its simplest form, providing:

  • Information about unique visitors, total visitors, page views, hits, etc.
  • Statistics about the most-accessed pages, workflows, and images
  • Statistics about the average (minimum, maximum, and percentile) number of visitors, concurrency over a period of time, and the busiest time of day or week for traffic
  • Details about average error rates across various pages and types of errors
  • Various types of users and user profiles
  • The most- and least-common user workflows and exit pages
  • Traffic patterns, trends, and average session duration of users
  • Statistics related to geographical distribution of users

Modern web-analytics systems providers like Omniture, Tealeaf, Google Analytics, and Webtrends primarily utilize onsite analytics. Some of them combine onsite and offsite analytics for even more detailed metrics collection and analysis. The following are some of the common web-analytics metrics:

  • Unique browsers and browser types
  • Number of new visitors
  • Visits, unique visitors, and unique authenticated visitors
  • Average number of visits per visitor
  • Page views and exits and site exits
  • Average number of page views per visit
  • Clicks, impressions, or hits
  • Average pages viewed per visitor
  • Ratio of new to returning visitors
  • Visits per month, quarter, or week
  • Conversion of non-subscribers to subscribers
  • Page views per visit by the active subscriber base
  • Visit or session duration
  • Average subscription length

About the author

Paul Fratellone's picture Paul Fratellone

Paul Fratellone is program director of quality and test consulting in the testing business unit of MindTree. Paul’s career of more than twenty-five years in information technology has been concentrated in testing, compliance, and quality management. He strives to achieve consistent execution to attain a predictable level of quality that is commensurate with the investment and enables leadership to objectively measure the success and continuous benefits from these investments.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!