Anonymous assessment – why and how?
It has been shown that a student’s prior good or poor performance might affect when grading unrelated assignments. The effect is called halo effect and it is one of the unconscious bias teachers need to be aware of. Knowledge about biases helps identify them and understanding creates an inclusive environment. In MyCourses it’s possible to hide student names and numbers while grading their work.
Our brains think both fast and slow
According to Daniel Kahneman’s work of two systems of cognition (known as dual processing theory), our brains think both fast and slow. These conceptual routes represent two different ways of processing information. How and why humans make decisions and why we make so many errors are the core questions in his theory. These decision-making processes are called system one, the intuitive system and system two, the deliberate system. (Kahneman, D. 2011, chapter 1).
System one is fast, always active, and ready to jump into action. It is based on heuristics and biases, taking mental shortcuts, and leaning towards associative memory. System two is thinking slow and needs effort and attention, such as consideration, assessment, and reflection. System two is the rational and logical part of our information processes but it is not tireless. And when it gets tired, system one kicks in. System one learns from experience and can make intuitive decisions when educated decision making is not possible. (Kahneman, D. 2011, chapter 1). Based on this we need to recognize the bias in decision making (Malouff, J.M. & Thorsteinsson, E.B. 2016, 245).
Bias in grading?
Bias can be either conscious or unconscious. Conscious bias is biased attitudes about certain groups, for example elderly people or politicians. There we know that we are being biased, and we are doing it intentionally. The focus in both biases, conscious or not, biases can be physical characteristics, such as ethnic background or gender. It can also be assigned status e.g., being classified as talented or a student with disability. When the bias comes from prior experience it is called halo bias. (Malouff, J.M. & Thorsteinsson, E.B. 2016, 245).
The possibility of conscious or unconscious bias in assessment leads to a recommendation to keep students anonymous whenever it is possible. (Malouff, J.M. & Thorsteinsson, E.B. 2016, 245).
According to Malouff, J.M et al, equity of assessment is a fundamental concern of all formal education. Research mentions universities that require keeping the students anonymous while grading their work whenever it is possible. (Malouff, J.M. et al, 2013, 233). This autumn faculty of law in the university of Helsinki moved into anonymous assessment in the pursuit of even more objective assessment. (Parikka, V. 31.8.2022).
Research on bias in grading has pointed out two expectancy-based biases: stereotyping and the halo effect. Stereotyping is commonly based on gender or ethnic background. The positive halo effect takes place when an initial favorable impression of an individual leads to higher evaluation later. It has been shown that student’s performance in oral presentation created bias and made a difference when grading student’s unrelated written work. (Malouff, J.M. et al, 2013, 233-236; Malouff, J.M. & Thorsteinsson, E.B. 2016, 245). Malouff et al. mention prior research that showed bias also related to student’s physical attractiveness, and prior assumptions of student being gifted (2013, 236).
Said findings do not offer explanation to bias phenomena but hypothesis is that all these biases involve implicit expectations about the quality of student’s performance. (Malouff, J.M. & Thorsteinsson, E.B., 2016, 252).
How to do anonymous grading in MyCourses?
Quiz
(Original text: Moodle Docs HY: Anonyymi arviointi tentti-aktiviteetissa)
Instructions in MCHelp Quizzes: Examples
After logging into MyCourses and to course’s workspace you need to change your role to ‘non-editing teacher’ from the user profile in the upper right-hand corner.
When your role is changed to Non-editing teacher, choose the quiz you want to grade and open the settings (gear) and choose ‘Manual grading’. This will hide students’ names.
After grading, go back to your user profile and choose ‘Return to my normal role’.
Assignment
(Original text: Moodle Docs: Anonyymi arviointi tehtävä-aktiviteetissa)
Instructions in MCHelp Assignment: Assignment settings (grade)
Anonymous submission is chosen in assignment settings (gear) in the Grade-section -> Anonymous submissions -> ‘Yes’.
Student submissions can be files or editor text (or both). Please give students instructions not to write their names or student numbers in files.
If you want that grades can be seen in the Grades section, you need to reveal the names again. One way to do this is to choose ‘View all submissions’ and Grading action: ‘Reveal student identities’.
Workshop
Instructions for anonymous peer grading can be found from MyCourses instructions Collaboration and peer review in MCHelp Workshop grading strategies and examples of grading.
Lastly
Assessment comes in many forms and the quiz and assignment in MyCourses do not serve it all. Rubric assessment would be one other option and it has been shown that it creates reliability. (Malouff, J.M. & Thorsteinsson, E.B., 2016, 253). You can read about using rubrics in blog post: How I reduced the time I spent explaining grades given to my students .
References
Kahneman, D. (2011). Thinking, fast and slow. Audiobook by Storytel.
Malouff, J. M., Emmerton, A. J., & Schutte, N. S. (2013). The Risk of a Halo Bias as a Reason to Keep Students Anonymous During Grading. Teaching of Psychology, 40(3), 233-237. https://doi.org/10.1177/0098628313487425
Malouff, J. M., & Thorsteinsson, E. B. (2016). Bias in grading: A meta-analysis of experimental research findings. The Australian journal of education, 60(3), 245-256. https://doi.org/10.1177/0004944116664618
Parikka, V. Oikeustieteen opiskelijat palauttavat jatkossa tenttinsä ilman omaa nimeä –”Pyritään yhä objektiivisempaan arviointiin”. Helsingin Sanomat, 31.8.2022.
Suveren. (2022). Unconscious Bias: Definition and Significance. Psikiyatride Güncel Yaklaşimlar, 14(3), 414–426. https://doi.org/10.18863/pgy.1026607
Additional reading material:
Aalto University Equality, Diversity and Inclusion Plan 2022-2024 (pdf)
Doornkamp, Van der Pol, L. D., Groeneveld, S., Mesman, J., Endendijk, J. J., & Groeneveld, M. G. (2022). Understanding gender bias in teachers’ grading: The role of gender stereotypical beliefs. Teaching and Teacher Education, 118. https://doi.org/10.1016/j.tate.2022.103826
Goldbach, Sickmann, J., & Pitz, T. (2022). Sequential decision bias – evidence from grading exams. Applied Economics, 54(32), 3727–3739. https://doi.org/10.1080/00036846.2021.1976390
