Chapter 10: Introducing evaluation

What to evaluate

There is a huge variety of interactive products with a vast array of features that need to be evaluated.

Focus on users and their tasks

Observe, measure, and analyze their performance with the system

Design iteratively

Why you need to evaluate

Evaluation is needed to check that users can use the product and like it.

Investing in user testing is a good thing.

Problems are fixed before the product is shipped.

The team can concentrate on real problems, not imaginary ones.

Engineers code instead of debating.

Time to market is sharply reduced.

Upon release, your sales department has a rock-solid design.

When to evaluate

Designers often develop mockups of the potential product that are used to elicit reactions from potential users.

The goal of evaluation is to assess how well a design fulfills users' needs and whether users like it.

Evaluations done during design to check that the product continues to meet users' needs are known as formative evaluations.

Early design ideas

HutchWorld is a virtual community developed through collaboration between Microsoft's virtual worlds research group and librarians and clinicians at thef red hutchinson cancer research center in Seattle, Washington.

This system enables cancer patients, their caregivers, family and friends to chat with one another, tell their storeis, discuss their experiences and coping strategies, among many other things.

Before developing the product, the team neded to learn about the patient experience at the Fred Hutchinson center.

They had to be particularly careful about doing this becauser many patients were very sick, they also typically go through bouts of low emotional and physical energy. Thus, it was clear from the start that developing a virtual community for this population would be challenging, but not impossible.

How was the testing done?

The team ran two main sets of user tests; the first set of tests was informally run onsite at the hospital.

After observing the system in use on computers located in the hospital setting, the team redesigned the software and then ran formal usability tests in the usability labs at MS.

Was it tested again?

Following the usability testing, there were more rounds of observation and testing with six new participants, two males and four females.

These tests followed the same general format as those just described but this time they tested multiple users at once, to ensure that the virtual world supported multiuser interactions.

The tests were also more detailed and focused; and this time the test results where more positive, but they obviously had problems that needed to be fixed.

Looking to the future

Future studies were planned to evaluate the effects of the computers and the software in the Fred Hutchinson Center.