Monthly Archives: November 2020

How to define a research question or a design problem

Posted on by

Introduction

Many texts state that identifying a good research question (or, equivalently, a design problem) is important for research. Wikipedia, for example, starts (as of writing this text, at least) with the following two sentences:

“A research question is ‘a question that a research project sets out to answer’. Choosing a research question is an essential element of both quantitative and qualitative research.” (Wikipedia, 2020)

However, finding a good research question (RQ) can be a painful experience. It may feel impossible to understand what are the criteria for a good RQ, how a good RQ can be found, and to notice when there are problems with some RQ candidate.

In this text, I will address the pains described above. I start by presenting a scenario of a project that has problems with its RQ. The analysis of that scenario allows me then to describe how to turn the situation described in the scenario for a better research or design project.

Scenario of a problematic project

Let us consider a scenario that you are starting a new research or design project. You have already an idea: your work will be related to communication with instant messaging (IM). Because you are a design-minded person, you are planning to design and develop a new IM feature: a possibility to send predefined replies on a mobile IM app. Your idea is that this feature will allow the user to communicate quickly with others in difficult situations where the they can only connect with others through their mobile phone. Your plan is to supply the mobile IM app with messages like “I’m late by 10 minutes but see you soon”, “I can’t answer back now but will do that later today”, and so on.

Therefore, your plan involves designing such an app, maybe first by sketching it and then illustrating its interaction with a prototyping software like Figma or Adobe XD. You may also decide to make your design functional by programming it and letting a selected number of participants to use it. These kinds of activities will let you demonstrate your skills as a designer-researcher.

Although predefined messages for a mobile IM app can be a topic of a great study, there are some problems with this project that require you to think more about it before you start. As the project is currently defined, it is difficult to provide convincing answers to these challenges:

  • Challenge 1: Why would this be a relevant topic for research or design? Good studies address topics that may interest also other people than the author only. The current research topic, however, does not do that self-evidently yet: it lacks an explanation why it would make sense to equip mobile IM apps with predefined replies. There is only a guess that this could be useful in some situations, but this may not convince the reader about the ingenuity of this project.
  • Challenge 2: How do you demonstrate that your solution is particularly good? For an outsider who will see the project’s outcome, it may not be clear why your final design would be the best one among the other possible designs. If you propose one interaction design for such a feature, what makes that a good one? In other words, the project lacks a yardstick by which its quality should be measured.
  • Challenge 3: How does this project lead to learning or new knowledge? Even if you can show that the topic is relevant (point 1) and that the solution works well (2), the solution may feel too “particularized” – not usable in any other design context. This is an important matter in applied research fields like design and human–computer interaction, because these fields require some form of generalizability from their studies. Findings of a study should result in some kind of knowledge, such as skills, sensitivity to important matters, design solutions or patterns, etc. that could be used also at a later time in other projects, preferably by other people too.

All these problems relate to a problem that this study does not have a RQ yet. Identifying a good research question will help clarify all the above matters, as we will see below.

Adding a research question / design problem

RQs are of many kinds, and they are closely tied to the intended finding of the study: what contribution should the study deliver. A contribution can be, for example, a solution to a problem or creation of novel information or knowledge. Novel information, in turn, can be a new theory, model or hypothesis, analysis that offers deeper understanding, identification of an unattended problem, description about poorly understood phenomenon, a new viewpoint, or many other things.

The researcher or thesis author usually has a lot of freedom in choosing the exact type of contribution that they want to make. This can feel difficult to the author: there may be no-one telling what they should study. In a way, in such a situation, the thesis/article author is the client of their own research: they both define what needs to be done, and then accomplish that work. Some starting points for narrowing down the space of possibilities is offered here.

Most importantly, the RQ needs to be focused on a topic that the author genuinely does not know, and which is important to find out on the path to the intended contribution. In our scenario about a mobile IM app’s predefined replies, there are currently too many alternatives for an intended contribution, and an outsider would not be able to know which one of them to expect:

  1. Demonstration that mobile IM apps will be better to use when they have this new feature.
  2. Report on the ways by which people would use the new feature, if their mobile IM apps would have such a feature.
  3. Requirements analysis for the specific design and detailed features by which the feature should be designed.
  4. Analysis of the situations where the feature would be most needed, and user groups who would most often be in such situations.

All of these are valid contributions, and the author can choose to focus on any one of them. This depends also on the author’s personal interests. This gives a possibility for formulating a RQ for the project. It is important to notice that each one of the possible contributions listed above calls for a different corresponding RQ:

RQ1: Do predefined replies in mobile IM apps improve their usability?

RQ2: How will users start using the predefined replies in mobile IM apps?

RQ3: How should the interaction in the IM app be designed, and what kind of predefined replies need to be offered to the users?

RQ4: When are predefined replies in IM apps needed?

This list of four RQs, matched with the four possible contributions, shows why the scenario presented in the beginning of this text was problematic. Only after asking these kinds of questions one is able to seek to answer to the earlier-presented three challenges in the end of the previous section. Also, each of the RQs needs a different research or design method, and its own kind of background research.

The choice and fine-tuning of the research question / design problem

Which one of the above RQs should our hypothetical researcher/designer choose? Lists of basic requisites for good RQs have been presented in many websites. They can help identify RQs that will still need refinement. Monash University offers the following kind of helpful list:

  1. Clear and focused. In other words, the question should clearly state what the writer needs to do.
  2. Not too broad and not too narrow. The question should have an appropriate scope. If the question is too broad it will not be possible to answer it thoroughly within the word limit. If it is too narrow you will not have enough to write about and you will struggle to develop a strong argument.
  3. Not too easy to answer. For example, the question should require more than a simple yes or no answer.
  4. Not too difficult to answer. You must be able to answer the question thoroughly within the given timeframe and word limit.
  5. Researchable. You must have access to a suitable amount of quality research materials, such as academic books and refereed journal articles.
  6. Analytical rather than descriptive. In other words, your research question should allow you to produce an analysis of an issue or problem rather than a simple description of it.

If a study meets the above criteria, it has a good chance of avoiding a problem of presenting a “non-contribution”: A laboriously produced finding that nonetheless does not provide new, interesting information. The points 3 and 6 above particularly guard against such studies: they warn the readers from focusing their efforts on something that is already known (3) and only describing what was done or what observations were made, instead of analysing them in more detail (6).

In fine-tuning a possible RQ, it is important to situate it to the right scope. The first possible RQ that comes to one’s mind is often too broad and needs to be narrowed. RQ4 above (“When are predefined replies in IM apps most needed?”), for example, is a very relevant question, but it is probably too broad.

Why is RQ4 too broad? The reason is that RQs are usually considered very literally. If you leave an aspect in your RQ unspecified, then it means that you intend that your RQ and your findings will be generalisable (i.e., applicable) to all the possible contexts and cases that your RQ can be applied to. Consider the following diagram:

Diagram shows 4 IM apps: WhatsApp, Facebook Messenger, Slack and Microsoft Teams. They belong to two sub-categories: leisure-oriented apps and work-oriented apps.

With a question “When are predefined replies in IM apps most needed?”, you are asking a question that covers both leisure-oriented and work-oriented IM apps which can be of very different kinds. Some of the IM apps are mobile-oriented (such as WhatsApp) and others are desktop-oriented (such as Slack or Teams). Unless you specify your RQ more narrowly, your findings should be applicable to all these kinds of apps. Also, RQ4 is unspecific also about the people that you are thinking as communication partners. It may be impossible for you to make a study so broad that it applies to all of these cases.

Therefore, a more manageable-sized scoping could be something like this:

RQ4 (version 2): In which away-from-desktop leisure life situations are predefined replies in IM apps most needed?

Furthermore, you can also narrow down your focus theoretically. In our example scenario, the researcher/designer can decide, for example, that they will consider predefined IM replies from the viewpoint of “face-work” in social interaction. By adopting this viewpoint, the researcher/designer can decide that they will design the IM’s replies with a goal that they help the user to maintain an active, positive image in the eyes of others. When they start designing the reply feature, they can now ask much more specific questions. For example: how could my design help a user in doing face-work in cases where they are in a hurry and can only send a short and blunt message to another person? How could the predefined replies help in situations where the users would not have time to answer but they know they should? Ultimately, would the predefined replies make it easier for users to do face-work in computer-mediated communications (CMC)?

You can therefore further specify RQ4 into this:

RQ4 (version 3): In which away-from-desktop leisure life situations are predefined replies in IM apps most needed when it is important to react quickly to arriving messages?

As you may notice, it is possible to scope the RQ too narrowly so that it starts to be close to absurd. But if that does not become a problem, the choice of methods (i.e., the research design) becomes much easier to do.

The benefit of theoretically narrowed-down RQs (in this case, building on the concept of face-work in RQ4 version 3) have the benefit that they point you to useful background literature. Non-theoretical RQs (e.g., RQ4 version 2), in contrast, require that you identify the relevant literature more independently, relying on your own judgment. In the present case, you can base your thinking about IM apps’ on sociological research on interpersonal interaction and self-presentation (e.g., Goffman 1967) and its earlier applications to CMC (Nardi et al., 2000; Salovaara et al., 2011). Such a literature provides the starting points for deeper design considerations. Deeper considerations, in turn, increase the contribution of the research, and make it interesting for the readers.

As said, the first RQ that one comes to think of is not necessarily the best and final one. The RQ may need to be adapted (and also can be adapted) over the course of the research. In qualitative research this is very typical, and the same applies to exploratory design projects that proceed through small design experiments (i.e., through their own smaller RQs).

Conclusion

This text promised to address the pains that definition of a RQ or a design problem may pose for a student or a researcher. The main points of the answer may be summarized as follows:

  • The search for a good RQ is a negotiation process between three objectives: what is personally motivating, what is realistically possible to do (e.g., that the work can be built on some earlier literature and there is a method that can answer to the RQ), and what motivates its relevance (i.e., can it lead to interesting findings).
  • The search for a RQ or a design problem is a process and not a task that must be fixed immediately. It is, however, good to get started somewhere, since a RQ gives a lot of focus for future activities: what to read and what methods to choose, for example.

With the presentation of the scenario and its analysis, I sought to demonstrate why and how choosing an additional analytical viewpoint can be a useful strategy. With it, a project whose meaningfulness may be otherwise questionable for an outsider can become interesting when its underpinnings and assumptions are explicated. That helps ensure that the reader will appreciate the work that the author has done with their research.

In the problematization of the scenario, I presented the three challenges related to it. I can now offer possible answers to them, by highlighting why a RQ can serve as a tool for finding them: 

  • Why would this be a relevant topic for research or design? Choice of a RQ often requires some amount of background research that helps the researcher/designer to understand how much about the problem has already been solved by others. This awareness helps shape the RQ to focus on a topic where information is not yet known and more information is needed for a high-quality outcome.
  • How do you demonstrate that your solution is particularly good? By having a question, it is possible to analyse what are the right methods for answering it. The quality of executing these becomes then evaluatable. The focus on a particular question also will permit that the author compromises optimality in other, less central outcomes. For example, if smoothness of interaction is in the focus, then it is easy to explain why long-term robustness and durability of a prototype may not be critical.
  • How does this project lead to learning or new knowledge? Presentation of the results or findings allows the researcher/design to devote their Discussion section (see the IMRaD article format) to topics that would have been impossible to predict before the study. That will demonstrate that the project has generated novel understanding: it has generated knowledge that can be considered insightful.

If and when the researcher/designer pursues further in design and research, the experience of thinking about RQs and design problems accumulates. As one reads literature, the ability to consider different research questions becomes better too. Similarly, as one carries out projects with different RQs and problems, and notices how adjusting them along the way helps shape one’s work, the experience similarly grows. Eventually, one may even learn to enjoy the analytical process of identifying a good research question.

As a suggestion for further reading, Carsten Sørensen’s text (2002) about writing and planning an article in information systems research field is a highly recommended one. It combines the question of choosing the RQ with the question on how to write a paper about it.

References

Goffman, E. (1967). On face-work: An analysis of ritual elements in social interaction. Psychiatry, 18(3), 213–231.  https://doi.org/10.1080/00332747.1955.11023008

Nardi, B. A., Whittaker, S., & Bradner, E. (2000). Interaction and outeraction: Instant messaging in action. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW 2000) (pp. 79–88). New York, NY: ACM Press. https://doi.org/10.1145/358916.358975

Salovaara, A., Lindqvist, A., Hasu, T., & Häkkilä, J. (2011). The phone rings but the user doesn’t answer: unavailability in mobile communication. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2011) (pp. 503–512). New York, NY: ACM Press. https://doi.org/10.1145/2037373.2037448

Sørensen, C. (2002): This is Not an Article — Just Some Food for Thoughts on How to Write One. Working Paper. Department of Information Systems, The London School of Economics and Political Science. No. 121.

Wikipedia (2020). Research question. Retrieved from https://en.wikipedia.org/wiki/Research_question (30 November 2020).

Typology of possible research findings (i.e., “contributions”)

Posted on by

Introduction

A good academic text delivers a clear and interesting message. That is often described as “contribution”. Good contributions teach something to the text’s readers: they change the reader’s way of thinking or acting, and increase their understanding and knowledge about an interesting subject. Thus, “a contribution is made when a manuscript clearly adds, embellishes, or creates something beyond what is already known” (Ladik & Stewart, 2008, p. 157). Such findings therefore present something that the researchers did not know so far; that is what makes the research article interesting.

Also a BA or a MA thesis deliver contributions. But their requirements for significant contribution are lower, as it is not crucial that a finding of a thesis should be a considered as a research contribution that generates novel understanding in the scientific community.

So, what are the possible findings and contributions that academic texts can make? Understanding what a contribution can be becomes easier over time, as one reads more literature and sees more examples of academic publications. But to get started more quickly, this article presents some classifications that I have found from other researchers’ writings, and finally presents a longer list that I have tailored for the fields of design and HCI.

The presentation of contributions in this article is in two parts: first I will discuss academic articles, and then I will add notes about BA/MA theses in HCI and design.

Types of findings and contributions in academic articles

Contributions can be classified along several dimensions. Some of the existing classifications are oriented to theoretical contributions. For example, Ladik and Stewart’s (2008) “contribution continuum”, written for a marketing research audience, divides the possible contributions to 8 classes, organized from minor to fundamental scientific impacts:

  1. Straight replications: studies that verify whether a finding that has been already published can be repeated.
  2. Replication and extension: similar to the one above, but with an adjustment.
  3. Extension of a new theory/method in a new area.
  4. Integrative review (e.g., meta-analysis).
  5. New theories to explain an old phenomenon, possibly also including a comparison between an existing and the new theory against each other to find out which one works better.
  6. Identifications of new phenomena worth of attention.
  7. Grand syntheses that integrate earlier theories together.
  8. New theories that predict new phenomena.

In addition to presenting the continuum, Ladik and Stewart’s (2008) text is great also in emphasizing many other characteristics of good academic texts too, such as a need to think about the target reader audience, need to emphasise surprise, and demonstrate passion and relevance of the topic that has been studied.

In human–computer interaction (HCI), which is more oriented to human-created objects, other kinds of contributions can be recognized. Wobbrock (2012) and Wobbrock and Kientz (2016) do not define the contributions based on their magnitude, but in terms of types of outcomes. As we can see, the theoretical contributions that were listed above are only one possibility in applied fields such as HCI and design:

  • empirical research findings (e.g., what factors and phenomena play an important role in different situations where people use technologies)
  • artefacts (i.e., designs and technologies)
  • methods
  • theories
  • datasets
  • surveys and reviews of existing research
  • opinions

In this classification, Wobbrock and Kientz’s papers themselves could be best classified as survey-like contributions, since their focus is on reviewing the kinds of research contributions in a research field as a whole. In this sence they synthesize together and explicate the practices in the field. In addition to being more directly useful also for HCI/design, Wobbrock’s suggestions are also great because both texts list papers from HCI research that exemplify these contribution types.

What is particular in the list above is the role of artifacts as research contributions. This is particular since it highlight’s HCI’s (and also design’s) nature as a “problem-solving” science (Oulasvirta and Hornbæk, 2016): in addition to producing the traditionally well-acknowledged empirical and conceptual (theoretical) contributions, HCI researchers also make constructive contributions by developing new technologies and designs.

Final distinction between contributions is their level of critical stance towards earlier research and practice. Most contributions are knowledge-increasing: they present new findings, expand the research to new areas, make existing theories and methods more detailed, accurate or more appropriate for some context, for example. These contributions are really common: with my colleagues we found, for example, that 94% of research papers in information systems research are knowledge-increasing (Salovaara et al., 2020). Many of the contribution classes presented in the lists above are like these too.

Other contributions are knowledge-contesting: they identify problems in the existing theories and methods, or in the practices by which they are used (Salovaara et al., 2020). They may also identify limits (“boundary conditions”) to the extent to which earlier contributions can be applied. In the spirit of science and research being a self-correcting process, the purpose of these knowledge-contesting contributions is to correct earlier mistakes in research and keep the research on the right track.

To summarise the considerations above, the following table presents a synthesis of possible contributions in HCI and design. A vast majority of the papers represent one (or sometimes several) of these contributions:

Contribution type Description Example
Boundary condition E.g., finding that there is a hard limit in some theory or a method that cannot be overcome by following the existing strategies. In mobile settings, users are usually not able to attend to their mobile phones more than 4 seconds at a time (Oulasvirta et al., 2005).
Demonstration of novel possibilities E.g., a new technology that has not been possible to build before Machine learning can help web designers develop better user interface layouts (Todi et al., 2016).
Extension to a new field E.g., adapting an approach to a new context What HCI research can offer for sexual wellbeing (Bardzell and Bardzell, 2011).
Falsification Demonstration that an accepted theory/belief is not true or has limited generalizability User interfaces should not be designed by having a premise that human behaviour is always planful. That is because of the situatedness of human action. Communication breakdowns between humans and technologies happen are inevitable, and assuming planful behaviour increases their severity (Suchman, 1987).
Incremental improvement Presentation of a solution that outperforms earlier approaches In mobile maps, it is better to visualize off-screen targets with triangle-like shapes (“wedges”) than with earlier-recommended arrows or radius-based circles (Gustafson et al., 2008).
Introducing another research field to one’s own field’s researchers, and showing that it can help in solving an interesting problem Presentation of a research field whose importance has not been noticed but which offers a lot of value Information foraging theory: that the theories about animals’ food-hunting can be applied to humans’ ways of searching for information in information spaces (Pirolli and Card, 1995).
New method Description of a method that is useful in many situations “Wizard of Oz” method: Futuristic technologies that cannot be built yet may be realistically studied if a human plays the computer’s part without the user’s awareness (Gould, 1983)
Novel concept E.g., an important phenomenon that needs to be remembered in future research and design ”Plausible deniability”: In human-human communication through a technology (e.g., a chat), technology should offer a possibility for humans to remain non-responsive without fear of “losing their face” (Nardi et al., 2000)
Sense-making E.g., an open-ended study that reports and analyses a phenomenon that is so far poorly understood What conversation strategies do online trolls use to derail and harm online conversations (Hardaker, 2013).
Synthesis / meta-analysis / review Summary of existing knowledge in the field Review of literature about the differences in novice and expert designers’ design processes (Cross, 2004).
Recommendations / guidelines Description by an experienced researcher or researcher team on how to use a certain method correctly. How qualitative data should be analysed using thematic analysis (Braun & Clarke, 2006).
Research agenda / manifesto Call for researchers to start addressing a neglected issue Call for attending to the widely spread misunderstanding and misuse of the term ”affordance” in much of design practice (Torenvliet, 2003).

This list is not comprehensive, and some areas have been covered in more detail than others. What is however notable in this list’s items is that papers about these contributions can be written using the same narrative format. That is because most of these contributions require a study: some method by which some material is analysed so that findings can be presented. Such papers can be readily written following the IMRaD-style narrative. Only the last two contributions – recommendations/guidelines and research agendas/manifestos – may need a different kind of a narrative and can therefore be harder to write well.

Examples of non-contributions in academic research

Notably, there are also certain types of papers that are often submitted for publication but which are often rejected and will therefore be rarely found in academic literature. When one is writing an article, it is a good idea to make sure that one is not writing one of those types of papers. Four common non-contributions are following:

  • Presentation of a well-designed system and its design process. These papers present well-designed systems and include evaluations that demonstrate the high quality of the outcome. The problem with these kinds of papers is that for a researcher looking for novel information, such papers offer very little to learn: they “only” describe well-conducted design process that already uses well-known methods. Only if these design processes solve hard problems in some contexts, and that these problems and their solutions generalise to other contexts too, the papers start to have value in terms of an academic contribution. That is because then the academic reader may conclude that the authors have found a way to address a problem that previously has been considered difficult to tackle. This kind of a study can be turned into an academic contribution by identifying a “design problem” that was solved in the process, and explaining why this problem is difficult and in what design situations similar problems can be encountered (i.e., where does the design problem and solution generalise to).
  • Case study report. Papers of this kind present observations or interview-based findings from field studies, and describe carefully methods that were used in these studies. A lot of effort may have been put into gathering all the data and to analyse it. Unfortunately, despite all the effort spent, also in this case, the conclusion by a reader may be that the story is interesting but lacks novelty: papers of this kind may be a well-conducted research projects but which only have applied rigorous methods without yielding novel findings. This kind of a study can be turned into an academic contribution by identifying an interesting and novel finding, and deepening the literature research so that it convinces the reader about the novelty and the need for this finding in the research field (e.g., a “research gap”).
  • Mappings of findings to a framework. Some papers present analyses from a complex settings and map these findings to a well-known theoretical framework (e.g. activity theory). The problem with such a finding is that it counts mostly as a demonstration that the framework can be used to make sense of observational data. This may not be surprising, if the same has been shown in numerous earlier studies too. This can be turned into an academic contribution, for example, by finding out that the framework cannot be used to make sense of some parts of the data, or that the framework needs adaptation because of the novel findings.
  • Landscaping and clustering studies without conclusions. Some automatic data analysis methods nowadays allow researchers to generate elaborate descriptive visualizations and groupings that can summarise complex phenomena in a neat manner. Examples of these methods include social network analysis, clustering methods of multidimensional data (e.g., factor analysis, k-means clustering and topic modeling), and sentiment analysis about natural language. If a paper only presents the outputs of such analyses, without identifying non-obvious patterns or conclusions, the paper easily lacks a clear contribution. An academic contribution would include an actionable message to the research field: a call for changing the research focus, or think about a common phenomenon in a new way. Typically this requires that the researchers interpret their clusters and identify something unexpected from them.

Contributions and findings in BA and MA theses in HCI and design

In BA and MA theses, the requirements are slightly different than in academic articles. The difference lies in the need for presenting a contribution vs another, more modest kind of a finding. A thesis does not need to demonstrate novelty to an entire research field; it only needs to demonstrate the ability to apply the relevant methods, theories and analytical thinking with respect to a meaningful problem of practical importance. Therefore the three last above-presented examples of non-contributions are, in fact, good candidates for excellent BA or MA theses even if they lack an academic contribution.

One may therefore conclude that in BA and MA theses, the goals can be more practically determined: They may orient to finding good designs or solutions for specific design problems. They may be reflections about the nature of a design process, such as explorations whether a certain design approach yields findings that satisfy the designer. They may also be oriented towards a designer or practitioner community than the researchers. Therefore they may deliver a call or message to those communities to start addressing issues or become aware of matters that are being neglected. Such issues do not need to relate to academic activities, but to societal issues, for example.

One or many contributions?

There can be one or many contributions in a paper. Some contribution types also go naturally together. For example, sometimes the most interesting contributions appear in the Discussion, after the answers to the research question(s) have already been presented. Thus a paper about an exploratory study may be sense-making in its Findings (e.g., by identifying an interesting underlying pattern or concept in the findings and by giving a name for it), but a manifesto-like contribution in its Discussion if it then shows how that concept may be crucial to remember in other situations too. Many readers may find that this manifesto-like contribution is actually more important than the text’s original finding.

However, many instructions on academic writing recommend that every text focuses on delivering only one “contribution”. For example, instructions published in Nature’s web page recommend to “Keep your message clear” (Gewin, 2018). There is a good reason for this: To offer a clear and interesting message, different contributions usually require different investigations. If one tries to combine several contributions together, they may require different methods, and these methods may conflict with each other, leading to biased and compromised results. Another problem is the need to reach a high clarity with the paper: if there are several intended contributions, explaining them clearly can be difficult. Jumping from talking about one contribution to another may be necessary, but this may confuse the reader. It is important to remember that it is the author’s responsibility to demonstrate that the findings are significant and interesting (e.g., Ladik and Stewart, 2008). Confusions should be avoided at all cost.

Conclusion

To conclude, to offer a clear contribution or a finding, it is a good idea to identify early on what kind of a story one wants to tell with their text. Following the recommendations of the IMRaD structure, for instance, all the attention of the paper’s argumentation can then be directed to delivering that message as clearly and convincingly as possible. This helps the readers – evaluators, reviewers, and others – appreciate the work that the author has done.

Acknowledgments

Thanks for Oscar Person for tipping me about Ladik & Stewart’s paper on research continuum.

References

Bardzell, J. & Bardzell, S. (2011). Pleasure is your birthright: Digitally enabled designer sex toys as a case of third-wave HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011) (pp. 257–266). New York, NY: ACM Press. https://doi.org/10.1145/1978942.1978979

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. https://doi.org/10.1191/1478088706qp063oa

Gewin, V. (2018). The write stuff: How to produce a first-class paper that will get published, stand out from the crowd and pull in plenty of readers. Nature, Vol. 555, pp. 129-130. Available at: https://media.nature.com/original/magazine-assets/d41586-018-02404-4/d41586-018-02404-4.pdf. Also available, with a different title, at https://www.nature.com/articles/d41586-018-02404-4 (retrieved 11 November 2020).

Cross, N. (2004). Expertise in design: An overview. Design Studies, 25(5), 427–441. https://doi.org/10.1016/j.destud.2004.06.002

Gould, J. D., Conti, J., & Hovanyecz, T. (1983). Composing letters with a simulated listening typewriter. Communications of the ACM, 26(4), 295–308.  https://doi.org/10.1145/2163.358100

Gustafson, S., Baudisch, P., Gutwin, C., & Irani, P. (2008). Wedge: Clutter-free visualization of off-screen locations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2008) (pp. 787–796). New York, NY: ACM Press.. https://doi.org/10.1145/1357054.1357179

Hardaker, C. (2013). “Uh….not to be nitpicky,,,, but…the past tense of drag is dragged, not drug.” – An overview of trolling strategies. Journal of Language Aggression and Conflict, 1(1), 58–86.  https://doi.org/10.1075/jlac.1.1.04har

Ladik, D. M. & Stewart, D. W. (2008). The contribution continuum. Journal of the Academy of Marketing Science, 36, 157–165.  https://doi.org/10.1007/s11747-008-0087-z

Nardi, B. A., Whittaker, S., & Bradner, E. (2000). Interaction and outeraction: Instant messaging in action. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW 2000) (pp. 79–88). New York, NY: ACM Press. https://doi.org/10.1145/358916.358975

Oulasvirta, A., Tamminen, S., Roto, V., & Kuorelahti, J. (2005). Interaction in 4-second bursts: The fragmented nature of attentional resources in mobile HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2005) (pp. 919–928). New York, NY: ACM Press. https://doi.org/10.1145/1054972.1055101

Oulasvirta, A. & Hornbæk, K. (2016). HCI research as problem-solving. In J. Kaye, A. Druin, C. Lampe, D. Morris, & J. P. Hourcade (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing (CHI 2016) (pp. 4956–4967). New York, NY: ACM Press.  https://doi.org/10.1145/2858036.2858283

Pirolli, P. & Card, S. (1995). Information foraging in information access environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1995) (pp. 51–58). New York, NY: ACM Press/Addison-Wesley. https://doi.org/10.1145/223904.223911

Salovaara, A., Upreti, B. R., Nykänen, J. I., & Merikivi, J. (2020). Building on shaky foundations? Lack of falsification and knowledge contestation in IS theories, methods, and practices. European Journal of Information Systems, 29(1), 65–83.  https://doi.org/10.1080/0960085X.2019.1685737

Suchman, L. A. (1987). Plans and Situated Actions: The Problem of Human–Machine Communication. Cambridge, UK: Cambridge University Press.

Todi, K., Weir, D., & Oulasvirta, A. (2016). Sketchplore: Sketch and explore with a layout optimiser. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (CHI 2016) (pp. 543–555). New York, NY: ACM Press. https://doi.org/10.1145/2901790.2901817

Torenvliet, G. (2003). We can’t afford it! The devaluation of a usability term. Interactions, 10(4), 12–17. https://doi.org/10.1145/838830.838857

Wobbrock, J. O. (2012). Seven Research Contributions in HCI. The Information School, DUB Group, University of Washington. http://faculty.washington.edu/wobbrock/pubs/Wobbrock-2012.pdf (retrieved 12 November 2020).

Wobbrock, J. O. & Kientz, J. A. (2016). Research contributions in human–computer interaction. Interactions (May–June), 38–44.  https://doi.org/10.1145/2907069