forward in applying computer technology to people’s lives.
3. An Internet Appliance Case Study
To move beyond generalizations, we introduce our Internet appliance case study. An ideal Internet appliance presents the features of the Internet, including e-mail and the World Wide Web, to the customer in a way that hides complexity, yet provides full access to the fun, fascinating, and convenient aspects. To enable this functionality seamlessly, the Internet appliance in our case study uses a client/server model. The servers at the ISP and the server farm provide e-mail services, customize and deliver news, weather, and other content to the appliance, filter undesirable Web sites, and provide the appliances with software updates. Figure one illustrates the architecture.
We consider this project a success for two reasons. First, our client has received positive reviews from critics, both technical (Peskovitz, 1999) and non-technical (Dreyfus, 1999). Second, the appliance is enjoying significant acceptance in the marketplace. While not perfect, the device is proving good enough for most customers, including many who have sent their first e-mail and surfed to their first Web site with it.
4. Loci of Internet Appliance Quality
As shown in figure one, we divide Internet appliance quality into three areas, which we call "loci of quality". We use this phrase because each area contains a set of behavioral points that determine the customer's experience of quality.
Intrinsic. Those aspects of quality that are strictly a function of the device hardware, the firmware, and the locally hosted software itself.
Systemic. Those aspects of quality that support the functions of the appliance, including the servers, the communications network, and the human aspects of the process.
Harmonic. Those aspects of quality that depend upon how well the otherwise-correct intrinsic and systemic functions interact with the Internet.
Moving from the intrinsic to the systemic to the harmonic locus of quality, the Internet appliance vendor controls fewer factors that influence quality. The vendor determines intrinsic quality just as a PC maker does. Systemic quality, while subject to some influence, depends considerably on the public switched telephony network, the quality of the server components, and the service levels of the ISP. For harmonic quality, the vendor must respond to the official and unofficial standards of the Internet, its Web servers, email applications, and its netizens.
5. Testing Techniques and Customer Quality Alignment
The testing process for Internet appliances is a straightforward evolution of standard test methodologies. The testing of the appliance itself resembles testing of a laptop computer, while one can test the server farm as one would test an information systems project at a bank. Like both of these situations, the prospective test manager will need to attend carefully to the logistics of the underlying hardware. Custom appliances, especially prototypes developed in off-shore facilities, are hard to come by early in the project as well as being temperamental, subject to failure, and difficult to maintain. Large server farms may contain hundreds of thousands of dollars worth of equipment. The first-time test manager for an Internet appliance project may want to refer to one author's book, Managing the Testing Process , for tips on how to handle these challenges (Black, 1999).
To plan our test effort, we started by uncovering the risks to product quality. Working cross-functionally in the organization, we put together a list of about 75 specific types of failure modes—such as "appliance won't boot," "slow e-mail transfers," etc.—in the following categories of quality risks:
- Error and disaster handling and recovery
- Capacity and volume
- Data flows and data quality
- States and state transitions
- Untested code
- Untested code