Tag Archives: usability testing
Clemmensen, T., Hertzum, M., Hornbaek, K., Shi, Q. and Yammiyavar, P. (2009) Cultural cognition in usability evaluation. Interacting with Computers, Vol. 21, No. 3, pp. 212-220.
DOI= 10.1016/j.intcom.2009.05.003 This article discusses on the cultural differences between Eastern and Western people in thinking aloud tests. Eastern people in this paper mean people with background from China or “countries heavily influenced by its culture” , and Western people … Continue reading
Isomursu, M., Kuutti, K. and Vainamo, S. (2004) Experience clip: Method for user participation and evaluation of mobile concepts. In Proceedings of the eighth conference on Participatory design: Artful integration: interweaving media, materials and practices (PDC 04), Vol. 1, pp. 83-92.
DOI=10.1145/1011870.1011881 In Experience Clip, a pair of users from the passers by is invited to participate in the evaluation of a mobile application in the use of which moving around is central. They gave the evaluated application to the other … Continue reading
Foelstad, A. and Hornbaek, K. (2010) Work-domain knowledge in usability evaluation: Experiences with Cooperative Usability Testing. The Journal of Systems and Software, Vol. 83, No. 11, pp. 2019-2030.
DOI= 10.1016/j.jss.2010.02.026 Foelstad and Hornbaek studied the use of Cooperative Usability Testing in the development of two work-domain specific systems. As modifications to the original method, they included an interpretation phase after each task, and used task-scenario walkthroughs instead of … Continue reading
Froekjaer, E. and Hornbaek, K. (2005) Cooperative usability testing: complementing usability tests with user-supported interpretation sessions. In CHI ’05 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’05), pp. 1383-1386.
DOI=10.1145/1056808.1056922 Froekjaer and Hornbaek present a usability testing method called Cooperative Usability Testing. It consists of two parts: The first part is an interaction session in which user interacts with the system as in contextual inquiry or in thinking aloud … Continue reading
Lindgaard, G. and Chattratichart, J. (2007) Usability testing: what have we overlooked?. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’07). ACM, New York, NY, USA, pp. 1415-1424.
DOI= 10.1145/1240624.1240839 The studies by Lindgaard and Chattratichart indicate a need to shift the focus from the number of test users to the number of test tasks in usability testing. Lindgaard and Chattratichart analysed the results of several usability teams … Continue reading
Holleran, P.A. (1991) A methodological note on pitfalls in usability testing. Behaviour & Information Technology, Vol. 10, no. 5, pp. 345-357.
DOI:10.1080/01449299108924295 Good usability testing is similar to good empirical research: the use of improper procedures will result in invalid data, and thereby poor validity and reliability. Holleran categorises pitfalls in usability testing into three groups: sampling problems mainly in planning … Continue reading
Kjeldskov, J., Skov, M.B. and Stage, J. (2004) Instant data analysis: conducting usability evaluations in a day. In Proceedings of the third Nordic conference on Human-computer interaction (NordiCHI ’04). ACM, New York, NY, USA, pp. 233-240.
DOI=10.1145/1028014.1028050 Kjeldskov et al. decided to test if the analysis phase of usability testing could be cut down and thereby cut the costs of testing. They utilized the resources already used in testing, i.e. the moderator and a note taker, … Continue reading
Hackman G.S. and Biers, D.W. (1992) Team Usability Testing: Are two Heads Better than One? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, October 1992; vol. 36, 16: pp. 1205-1209.
DOI= 10.1177/154193129203601605 Hackman and Biers made studies to compare the performance of a single user alone, a single user with an observer and a pair of users all using the thinking aloud method. Their results showed that the presence of … Continue reading
Höysniemi, J., Hämäläinen, P. and Turkki, L. (2003) Using peer tutoring in evaluating the usability of a physically interactive computer game with children. Interacting with Computers, Vol. 15, No. 2, pp. 203-225.
DOI= 10.1016/S0953-5438(03)00008-0 This study used peer tutoring to evaluate an interactive computer game with children. They used either a pair of children or one child at a time to teach the use of the game to another child. This way, … Continue reading
Kennedy, S. (1989) Using video in the BNR usability lab. SIGCHI Bulletin. Vol. 21, No. 2, pp. 92-95.
DOI=10.1145/70609.70624 Co-discovery learning shares many principles with constructive interaction, but in addition, has a list of spesific tasks and includes a reflection on the task difficulty after each task. Sue Kennedy and her colleagues used this method in evaluating various … Continue reading
O’Malley, C.E., Draper, S.W. & Riley, M.S. (1984) Constructive interaction: A method for studying human-computer-human interaction. In Shackel, B. (Ed.) Human-computer interaction – INTERACT’84. pp. 269-274.
Constructive interaction is a method involving two users at the same time in solving a problem. O’Malley et al. brought this method into the studies of human-computer interaction in the mid 1980’s. In this method, two subjects with comparable expertise … Continue reading
van den Haak, M.J. and de Jong, M.D.T. (2005) Analyzing the interaction between facilitator and participants in two variants of the think-aloud method. Proceedings of the International Professional Communication Conference, 2005 (IPCC 2005). pp. 323- 327.
DOI= 10.1109/IPCC.2005.1494192 Van den Haak and de Jong (2005) compared the interaction between the test moderator and test user in two different settings: using thinking-aloud method alone and paired user testing that they call constructive interaction test. They analysed parts … Continue reading
Sauro, J. and Lewis, J.R. (2009) Correlations among prototypical usability metrics: evidence for the construct of usability. In Proceedings of the 27th international conference on Human factors in computing systems (CHI ’09). ACM, New York, USA, pp. 1609-1618.
DOI= 10.1145/1518701.1518947 To study the correlations between common usability metrics used in typical usability tests, Sauro and Lewis made an analysis focusing on summative usability studies made in practice. They were able to collect data from 90 usability tests in … Continue reading
Nielsen, J. (1993) Usability Engineering. Academic Press, Boston, USA. 358 p.
Although Nielsen’s book is now 20 years old, it still is a good reference on things like how to make a heuristic evaluation, how can usability be measured and what sort of ethical considerations are involved in having real users … Continue reading
Kjeldskov, J., Skov, M.B. and Stage, J. (2005) Does time heal? A longitudinal study of usability. In Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future (OZCHI ’05). Computer-Human Interaction Special Interest Group (CHISIG) of Australia, Narrabundah, Australia, Australia, pp. 1-10
Kjeldskov, Skov and Stage approached the question of differences between novice and expert users as test participants in a new way. Instead of having different users representing either novice or expert users, they had the same seven users as test … Continue reading
Hornbaek, K. and Law, E.L-C. (2007) Meta-analysis of correlations among usability measures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07). ACM, New York, NY, USA, pp. 617-626.
DOI= 10.1145/1240624.1240722 Hornaek and Law repeated a similar meta-analysis as Nielsen and Levy (1994) on usability studies reported in top HCI conferences and journals. They studied the data from 73 usability studies and analysed correlations between various usability measures, such … Continue reading
Raita, E. and Oulasvirta, A. (2011) Too good to be bad: Favorable product expectations boost subjective usability ratings. Interacting with Computers, Vol. 23, No. 4, pp. 363-371.
DOI= 10.1016/j.intcom.2011.04.002 This article studies the effect of positive or negative priming on the subjective usability ratings after the test. The priming was done with two different versions of product review given to the users before starting with the test … Continue reading
Nielsen, J. and Levy, J. (1994) Measuring usability: Preference vs. performance. Communications of the ACM, Vol. 37, no. 4, pp. 66-75.
DOI= 10.1145/175276.175282 Nielsen and Levy (1994) made a meta-analysis to study the correlation between test users’ performance with the compared products and their preferences after the test. They found a strong positive correlation between performance and preferences, but several cases … Continue reading
Hartmann, J., Sutcliffe, A. and De Angeli, A. (2008) Towards a theory of user judgment of aesthetics and user interface quality. ACM Transactions on Computer‐Human Interaction, Vol. 15, No. 4, Article 15, pp. 15:1-15:30.
DOI=10.1145/1460355.1460357 Also this article contributes to the discussions on the interplay between aesthetics and perceived usability. Hartmann et al. used two versions of two web sites in their studies. A metaphor-based design was more aesthetic and also more engaging than … Continue reading
Cockton, G. & Woolrych, A. Sale Must End: Should Discount Methods be Cleared off HCI’s Shelves? Interactions, September+October 2002. 13-18
Interactions’ article is a critical analysis of the so called discount usability methods. By discount methods, the authors mean almost all usability inspection methods that I consider business as usual in industry, i.e. testing with only 3-5 users, heuristic evaluations … Continue reading
