Requirements

Conference Presentations

STAREAST 2011: Automating Embedded System Testing

Many testers believe that it is prohibitively costly and time-consuming to automate embedded and mobile phone application testing. By approaching the problem from a test design perspective and using that design to drive the automation initiative, Hans Buwalda demystifies automated testing of embedded systems. He draws on experiences gained on a large-scale testing project for a leading smart-phone platform and a Window CE embedded automotive testing platform. Hans describes the technical side of the solution-how to setup a tethered automation agent to expose the GUI and drive tests at the device layer. Learn how to couple this technology solution with a test design methodology that helps even non-technical testers participate in the automation development and execution. Take back a new approach to achieve large-scale automation coverage that is easily maintainable over the long term.

Hans Buwalda, LogiGear
Testing Embedded Software Using an Error Taxonomy

Just like the rest of the software world, embedded software has defects. Today, embedded software is pervasive-built into automobiles, medical diagnostic devices, telephones, airplanes, spacecraft, and really almost everything. Because defects in embedded software can cause constant customer frustration, complete product failure, and even death, it would seem critical to collect and categorize the types of errors that are typically found in embedded software. Jon Hagar describes the few error studies that have been done in the embedded domain and the work he has done to turn that data into a valuable error taxonomy. After explaining the concept of a taxonomy and how you can use it to guide test planning for embedded software, he discusses ways to design tests to exploit the taxonomy and find important defects in your embedded system.

Jon Hagar, Consultant
The Power of the Crowd: Mobile Testing for Scale and Global Coverage

Crowdsourced testing of mobile applications, a middle ground between in-house and outsourced testing, has many advantages: scale, speed, coverage, lower capital costs, reduced staffing costs, and no long-term commitments. However, crowdsourced testing of any application-mobile or not-should augment your professional testing resources, not replace them. Most importantly, crowdsourced testing has to be done well or it’s a waste of time and money. John Carpenter reviews the applications and ways he’s outsourced testing to the crowd. Focusing on adopting crowdsourcing for both functional and usability testing in mobile applications, John describes scenarios in which you can leverage the crowd to lower costs and increase product quality, including scaling the application to large populations of global users.

John Carpenter, Mob4Hire, Inc.
Test as a Service: A New Architecture for Embedded Systems

The classic models adopted in test automation today-guaranteeing ease of test implementation rather than extendibility of the test architecture-are inadequate for the unprecedented complexity of today’s embedded software market. Because many embedded software solutions must be designed and developed for multiple deployments on different and rapidly changing hardware platforms, testers need something new. Raniero Virgilio describes a novel approach he calls Test as a Service (TaaS), in which test logic is implemented in self-consistent components on a shared test automation infrastructure. These test components are deployed at runtime to make the test process completely dynamic. The TaaS architecture provides specific high-level test services to testers as they need them.

Raniero Virgilio, Intel
Testing with Virtual Machines: Past, Present, and Future

In the past several years, virtualization has dramatically improved tester productivity. A virtual machine is a useful abstraction for encapsulating the entire software stack. Roussi Roussev presents proven techniques that no modern test environment is complete without. Running multiple virtual machines on a single host maximizes hardware resource utilization and reduces operating costs. Strong isolation facilitates building security testing and multi-tenant environments. With the help of snapshots, virtual machines can quickly travel in time and space. Virtual hardware makes simulating machine, cluster or entire datacenter failure scenarios a whole lot easier. Deterministic record/replay helps track hard to reproduce bugs, comparing outputs allows for measuring the impact of small configuration and binary changes.

Roussi Roussev, VMware
STARWEST 2010: Automating Embedded System Testing

Many testers believe the challenges of automating embedded and mobile phone-based systems testing are prohibitively difficult. By approaching the problem from a test design perspective and using that design to drive the automation initiative, William Coleman demystifies automated testing of embedded systems. He draws on experiences gained on a large-scale testing project for a leading smart-phone platform and a Window CE embedded automotive testing platform. William describes the technical side of the solution-how to setup a tethered automation agent to expose the GUI and drive tests at the device layer. Learn how to couple this technology solution with a test design methodology that helps even non-technical testers participate in the automation development and execution. Take back a new approach to achieve large-scale automation coverage that is easily maintainable over the long term.

William Coleman, LogiGear Corporation
Using the Amazon Cloud to Accelerate Testing

Virtualization technologies have been a great boon to test labs everywhere. With the Amazon Elastic Compute Cloud (EC2), these same benefits are available to everyone-without the need to purchase and maintain your own hardware. Once you master the tricks and tools of this new technology, you too can instantly have limitless capacity at your disposal. Randy Hayes demonstrates how to use the AWS Management Console to create virtual test machines (AMIs), use S3 storage services, handle elastic IP Addresses, and leverage these services for functional testing, load testing, defect tracking, and other common testing functions. Randy explains the Amazon Virtual Private Cloud, which allows EC2 cloud instances to be configured to run inside your firewall to test inward-facing applications. Gain access to a pre-configured AMI with open source testing tools and other utilities for quickly migrating your test lab to EC2.

Randy Hayes, Capacity Calibration, Inc.
Exploratory Testing of Mobile Applications

Exploratory testing-the process of simultaneous test design, execution, and learning-is a popular approach to testing traditional application software. Can you apply this approach to testing mobile applications? At first, it is tempting to merely employ the same methods and techniques that you would use with other software applications. Although some concepts transfer directly, testing mobile applications presents special challenges you must consider and address. Jonathan Kohl shares his experiences with testing mobile apps, including the smaller screens and unique input methods that can cause physical strain on testers and slow down the testing effort. Smaller memory and less processing power in the device mean tests often interfere with the application’s normal operation. Network and connectivity issues can cause unexpected errors that crash mobile apps and leave testers scratching their heads.

Jonathan Kohl, Kohl Concepts, Inc.
Transform Your Lifecycle-Virtualize the Test Lab

Every tester has heard "it works on my machine" from a developer, referring to a defect deemed to be non-reproducible. We all know the back-and-forth conversations and have yearned for ways to easily replicate test environment failures in the development environment. Test organizations often struggle with access to test environments that closely match production while the operations department struggles to keep up with the demand for provisioned environments. Virtual lab technology can solve these frequent, tedious, and expensive problems, delivering immediate productivity and return-on-investment. By shattering barriers between development, testing, and operations, virtual lab technology is transformational and promises to be the hub of the modern application lifecycle. Theresa Lanowitz shares the results of the "voke Market Snapshot" report on virtual lab management.

Theresa Lanowitz, voke, Inc.
Crowdsourced Testing of Mobile Applications

With new mobile applications for Blackberry, iPhone, and Android battling for media attention and consumer dollars, the pressure to get applications built, tested, and launched has never been greater. Getting high-quality apps to market quickly can make or break a product or company. However, the testing methods that work for Web and desktop apps (e.g., in-house QA, outsourcing, emulators/simulators, and beta testers) do not meet the extreme testing needs of mobile apps. Companies must test across many handset makers and models, wireless carriers, operating systems, browsers, and locations. This calls for a new approach-crowdsourcing. Doron Reuveni provides insight into the growing trend of crowdsourced testing for mobile applications and addresses both the benefits and challenges of this new testing model.

Doron Reuveni, uTest

Pages

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.