Six years ago, not long after the Sarbanes-Oxley Act introduced new levels of oversight to public companies, Linda Hayes speculated about what the legislation might mean for the state of software testing in large, public corporations. "Software QA is no longer an optional function primarily designed to protect developers from their mistakes," Linda wrote, "but is an essential one that protects them from SEC sanctions, civil damages, and an all-expense paid vacation to Club Fed." Now, she takes another look at her own forecast and how Sarbanes-Oxley has changed the testing environment.
When the Sarbanes-Oxley Act ("SOX") was passed in the wake of Enron and other examples of bad business behavior, I wrote a column for StickyMinds "Hello Up There! The Sarbanes Effect" speculating that maybe—just maybe—this legislation would elevate that status of testing in the corporation. After all, if corporate officers and directors had to accept potential liability for errors or omissions, they might look at testing as more of a necessity and less of a luxury. What a breakthrough that would be, from the bowels of IT to the rarefied air of mahogany row!
It may be coming true. In the past year, I have experienced more than one instance of corporate audit and compliance inserting itself into the testing area and vice versa. This is unprecedented in my experience and may portend the anointing of testing as the new accounting of IT.
Testing in the Audit Trail
In one case, the audit group declared itself a stakeholder in the user acceptance testing process and promoted a specific agenda around the goals and deliverables, requiring sign-off on test strategies, plans, test cases, and results. In another case, the compliance group performed an independent, internal survey to ascertain whether protected identity information was being exposed during test. In both cases the test organization was a direct beneficiary of the attention.
But the most interesting—and promising—case was when an enterprising manager was able to displace hundreds of thousands in annual audit fees by leveraging her automated test tools and processes and applying them to SOX. Having previously been a formal external auditor, she helped lead her own organization through year two of SOX. She gained hands-on knowledge and experience with the challenges of getting the business to test SOX controls. She and her team also had been heavily involved in transferring many of the first-year manual controls to application controls. Now, as the global testing director for a $4 billion manufacturing concern with a very complex, global SAP system, she saw on a daily basis how the impact of change could also impact the SOX-controlled environment they had put in place.
Thorough testing of new projects, as well as managing testing of upgrades and enhancements to the existing production environment, required the execution of hundreds of tests was almost a full time job for many of the business owners. In order to take off some of the manual testing effort and get back accurate results more quickly, she had already made the business case to automate the regression baseline testing, which covered over 400 of the most highly used transactions and a solid base of their critical processes. With the right combination of technology, skills, and effort and working together with the business, her team was able to automate and validate the more than 400 transactions and associated processes using seventy-five end-to-end scripts across seven primary functional areas. Pretty impressive.
Having tackled that elephant and looking at the changes yet to happen, she had an inspiration. There were over seventy individual financial controls managed by the software that had to be audited. Either the business had to execute the tests and the audit firm retest or a third party could be retained to independently test, but either option would cost more than $250,000 per year, every year, plus hundreds of internal man hours. Working with the external auditors and internal compliance team, she showed them how they could get higher quality results using an automated approach and still satisfy the requirements for independent validation of the software's internal controls.