Kjeldskov, J., Skov, M.B. and Stage, J. (2005) Does time heal? A longitudinal study of usability. In Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future (OZCHI ’05). Computer-Human Interaction Special Interest Group (CHISIG) of Australia, Narrabundah, Australia, Australia, pp. 1-10
Kjeldskov, Skov and Stage approached the question of differences between novice and expert users as test participants in a new way. Instead of having different users representing either novice or expert users, they had the same seven users as test participants with one year break during which the users became experts. In the first tests, the system was just introduced in the users’ workplace, and the second test took place only after the test users had used the system on daily basis over a year.
The results of Kjeldskov et al. showed that the expert users solved significantly more tasks than the novices, but still, there were no significant differences in the total task completion times. On the part of number of usability problems, the novice users found significantly more problems than the experts, and the novices assessed the problems as more severe than the experts. Several problems remained the same even though the users had become experts, but as expert users they rated these problems less severe. Both novice and expert users also found unique problems that the other group did not encounter. Additionally, novice users experienced significantly more mental workload and frustration in the post-test workload measurement.