Accession Number= 00000487-196211000-00005
Finally I found an article presenting notes on social psychology studies regarding the behavior of people as test participants. Although usability tests are not generally treated as scientific or psychological experiments, many similar phenomenon as in Orne’s studies can be identified also in usability tests. For example, the test users are ready to “perform a very wide range of actions on request without inquiring as to their purpose, and frequently without inquiring as to their durations”, as Orne mentions. Furthermore, the tasks are carried out with care even for hours until the experimenter ends the session. It is unlikely that the test users would perform with similar care and persistence in contexts outside the tests.
In Orne’s studies, test participants have been shown to be eager to validate the experimental hypothesis that they infer from various cues ranging from the reputation of the research filed, rumors of the specific study and research group to the gestures and tones of the experimenter. Even if the tasks are clearly meaningless, the participants continue to do them and rely the test to have an important purpose. As the factors affecting to test participants’ expectations and behaviour cannot be eliminated, Orne recommends to identify them as systematically as possible to be able to assess their effect on the results and especially the extent to which the results can be generalized to contexts outside test situations.
Although the article is even older than me, people have not changed, so they behave much the same way in usability tests nowadays, so it is important to have at leas some idea of the numerous factors possibly influencing the test.