This article discusses on the cultural differences between Eastern and Western people in thinking aloud tests. Eastern people in this paper mean people with background from China or “countries heavily influenced by its culture” , and Western people refer to people in Western Europe and US citizens with European background. (This leads to a question: where do Finns stand?)
The paper does not include any own, original study, but it summarizes nicely results from other studies related to cultural differences and testing. The paper also gives quite a nice and simple model of the factors of communication in a thinking aloud test, namely
– the system with which the user interacts
– the user verbalizing his/her thoughts
– the instructions and tasks given to the user to give a frame for the actions
– the evaluator making interpretations of the user’s actions and comments
– a list of problems found made by the evaluators
This model gives several points in which cultural differences may make the difference and lead to wrong interpretations or unexpected actions. For example, the Eastern users do not give negative feedback if they do not know the evaluator well enough, and the process of thinking aloud is so difficult to Eastern people overall that it impairs their performance with the tasks. With Western users, some studies show that thinking aloud even improves their performance compared to working in silence.
The article refers to several articles studying the effects of culture in people’s behavior, especially in test situations. An interesting example is that Eastern people pay more attention to the context of situations and happenings, whereas Western people tend to focus on the people involved in the situation and their actions. This leads to the thing that was at this point the most relevant for me in this paper, i.e., at least Eastern users need a scenario for the test tasks to be able to understand what they are supposed to do and why. For Western people, tasks can be given even as a list of separate tasks without any background or scenario, and this is ok for them.
This again leads to an interesting question: where do Finns stand in this issue about scenarios? In my experience, I think we stand somewhere in the middle having effects from both sides, and I do not know if age, for example, has some effect in this matter. Might be worth studying…