Heuristics in Software Testing and Usability Evaluation
Heuristic evaluation is one of the well known approaches for usability evaluation. The goals of manual software testing are closely related to usability evaluation. Thus, it seems quite plausible that heuristics could be one practical way of applying certain types of knowledge in software testing. In practice, the heuristics seem to be commonly used in testing. Practitioners find different types of heuristics and plain checklists very useful to guide and organize their testing. Actually, many testing practitioners and consultants have shared their heuristics for others. Here are some examples:
– Heuristics as test oracles, how do I recognize a failure when I see it
- Hoffman, D. (1999) “Heuristic Test Oracles“, STQE Magazine
- Bolton, M. (2005) “Testing Without a Map“, Better Software Magazine,
– Heuristics for selecting relevant test cases
- Hendrickson, E. “Test Heuristics Cheat Sheet“
– Risk based and test strategy heuristics
– Tour heuristics
The testing heuristics are grown in practice and are often not founded to theoretical background or empirical research. It also seems that test heuristics can be very context specific and are often proposed to be individually tailored rather than meant to apply very generally. Test heuristics are sometimes just a checklist that doesn’t provide much guidance on how or when to apply them. The concept of heuristics is used in many different ways on varying levels of abstraction in testing.
These examples of the use of testing heuristics in practice raise some interesting research questions, so I thought there has to be a lot of research going on around the use of heuristics in software testing. Surprisingly, my searches for heuristics in software testing in scientific databases gives results on automatic test case generation; applying genetic programming, and other AI approaches to testing; automated testing of heuristic systems; search based testing; etc. But nothing on the actual use of heuristics in testing or heuristic testing methods! This is often the result when I try to search for research on something that I find interesting and relevant in the field of software testing. I think heuristic testing could learn something from the research done on the use of heuristics in usability evaluations.