(click here for a printable version)
Insights from Human Factors International, Inc. (HFI) Providing consulting and training in software ergonomics. (http://www.humanfactors.com/)
Every month HFI reviews the most useful developments in UI research from major conferences and publications.
In this issue:
Dr. Bob Bailey asks -- How good are designers at predicting user performance?
The Ergonomic Pragmatist, Dr. Eric Schaffer, gives practical advice.
Bob Bailey, Ph.D., Chief Scientist for HFI
A few years ago, two studies were published that showed clearly how much trouble designers had when trying to predict human performance in their systems (Bailey, 1993; Tullis, 1993). More recently, Miika Silfverberg, Scott MacKenzie and Panu Korhonen at the Nokia Research Center in Finland conducted an interesting study comparing two devices. They had users enter text information into cell phones. One group entered the information using "one-handed thumb input," where subjects held the phone in one hand and pressed the keys with the thumb of the same hand. The other group used "two- handed index finger input," where participants held the phone in their preferred hand and pressed the keys with the index finger on their other hand.
Which way do you think is fastest? A group of 101 website designers and usability professionals were asked to indicate which would elicit the fastest performance (Bailey, 2001). Twenty-three percent voted for the "thumb input" and 77% thought the "two-handed input" would be faster. The Nokia study showed that the two-handed approach was 12% faster than one-handed. In this case, over 3 out of 4 participants guessed correctly.
A second study compared entry speed using the two-handed approach with the "T9" approach. T9 is produced by Tegic Communications in Seattle. T9 uses a dictionary to disambiguate the letters when entering text into a cell phone. For example, to key "t-h-e" users pressed 8-4-3-0 (the 0-key is used as a SPACE). T9 then compares the word possibilities to its linguistic database to "guess" the intended word. It shows the most common word as a default, and if users want to select another word, they press the star key for NEXT.
The same group of website designers and usability professionals were asked to indicate which would elicit the fastest performance -- the normal (traditional) way of entering letters into a cell phone or the T9 approach. Fifty-eight percent voted for the "normal" method and only 42% thought the T9 method would be faster. The Nokia study showed that T9 was 35% faster than the "normal" method. T9 was amazingly fast, allowing entry speeds of 33 words-per-minute, compared with only 21 wpm for the "normal" approach. In this case, over half of the participants missed the essential cues for making the correct judgment.
Having designers guess the best way of achieving optimal user performance is very difficult. Their design decisions can be improved by ensuring that designers are familiar with the research literature, and by effectively using performance-based usability testing.
References:
Bailey, R.W. (2001), Annual User Interface Update - 2001.
Bailey, R.W. (1993), Performance vs preference, Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, 282-286.
Silfverberg, M., MacKenzie, I.S. and Korhonen, P. (2000), Predicting text entry speed on mobile phones, Proceedings of CHI 2000, 9-13.
Tullis, T.S. (1993), Is user interface design just common sense? Proceedings of the 5th International Conference on Human- Computer Interaction, 9-14.
The Ergonomic Pragmatist Eric Schaffer, Ph.D., CPE, Founder and CEO of HFI
Actually, there ARE times to use an educated guess and times to get data. The professional knows the difference. The POINT of running studies is to reveal things that are not obvious. Otherwise the research is what we derisively call a "Grandmother" study (you could just ask your Grandmother and SHE will tell the answer). 95% of the usability questions I see ARE obvious to a professional in the field. They may be obvious by common sense. They may be obvious because we have well-learned principles and models of human behavior. They may be obvious after reading thousands of studies and guessing the result... then seeing what really happened.
The pragmatic question is, how many usability professionals would guess the result of these two studies and say they were very sure of being correct. Hopefully pretty few. They are both interesting studies. I was not sure which would be best myself. So I would know to RUN these studies, if the difference was important. But if you want to research whether putting your text in all capital letters will effect reading speed: save your money - we KNOW the answer.
3-day Annual User Interface Update Seminar presented by Dr. Robert Bailey. http://www.humanfactors.com/training/annualupdate.asp.
REGISTER for Bob's UI Update Seminar:
The schedule for 2002 seminars can be downloaded at http://www.humanfactors.com/downloads/2002calendar.pdf
Suggestions, comments, questions? HFI editors at mailto:hfi@humanfactors.com.
Want past issues? http://www.humanfactors.com/library/pastissues.asp
Subscribe? - http://www.humanfactors.com/library/subscribe.asp
Do NOT want this newsletter? E-mail mailto:unsubscribe@humanfactors.com with a Subject of: "Unsubscribe Newsletter"