Triangulation in UX Research
What is it and how can you use it to improve the quality of your user insights?
Triangulation is becoming a buzzword in UX research. What is it and how can you use it to improve the quality of your user insights?
The term comes from geometry (trigonometry) and refers to the process of determining the location of a point by forming triangles to the point from known points. In social sciences, triangulation refers to using multiple research methods to study the same phenomenon (e.g., interviews and quantitative surveys). The concept is similar to that of cross-examination in forensic science. By asking multiple witnesses we get closer to the truth. That happens by uncovering consistencies and inconsistencies in the data.
According to the Encyclopedia of Research Design:
Triangulation is the practice of using multiple sources of data or multiple approaches to analyzing data, to enhance the credibility of a research study.
In research credibility usually refers to:
internal validity (or credibility in qualitative research): are we actually measuring what we’re intending to measure?
external validity (or transferability in qualitative research): is what we’re measuring applicable to the real world?
reliability (or dependability in qualitative research): if we did this study again, would we get the same results?
According to Denzin no single method, theory, or observer can capture all that is relevant or important in a study. Triangulation was offered as a solution to this. Denzin in the 1970s identified 4 main types of triangulation, which are still widely recognized by the research community:
Data triangulation (multiple datasets): this refers to using a variety of data sources in a study. In particular, it involves gathering data through differing sampling strategies such as collecting data at different times, in different contexts, or from different people. This approach allows weaknesses in the data to be compensated for by the strengths of other data, thus increasing the validity and reliability of the results.
Investigator triangulation (multiple researchers): this method involves using several different investigators/evaluators in the same investigation. In order to triangulate, each different evaluator would conduct the same examination using the same method (e.g., interview, observation). The findings from each evaluator would then be compared. If all researchers arrive at the same conclusion, then validity is established. In UX research this could be achieved by having multiple researchers analyze the same set of qualitative data. Using researchers from different ethnic, age, gender, and class groups can be used to check for things like observer and interviewer bias.
Theory triangulation (multiple theories): this method only involves one data set but the researcher applies different theories and alternative theories to it. Particularly, one views the data through a theoretical lens and through contradictory theories.
Methodological or method triangulation (multiple methods): It refers to the use of multiple methods to study a situation or phenomenon. The purpose of this approach is to make up for the deficiencies and biases of one method by using the strengths of another. This type of triangulation is very similar to the mixed-method approaches used in social science research, where the results from one method are used to enhance, augment and clarify the results of another.
Triangulation in User Research
Different methods answer different questions but also have limitations. No method in user research (or any kind of social research) is perfect. Stakeholders and junior researchers often ask me how we can address the limitations of various research methods and my answer is usually… triangulate! Even in well-planned, properly controlled research there will be limitations. Using multiple perspectives (methods, data points, researchers, or theories) about a specific problem, will increase our confidence and help us achieve credibility (Fusch et al., 2018).
In UX research the most common methods of triangulation are method triangulation (e.g., using a survey and conducting interviews to study the same problem) and investigator triangulation (e.g., multiple researchers analysing the same dataset). Triangulation allows us to have more confidence in the research data, can reveal unique unexpected findings, and allows us to understand a phenomenon more clearly.
Examples of triangulation in User Research:
Using multiple methodologies for the same question in order to understand the why and how of user behaviour. For example, we can use a survey to see how people use a specific feature and follow up with in-depth interviews to understand why. Using mixed methods often gives a clearer perspective of the problem
Often, research will begin with a qualitative method to identify and narrow down the problem. For example, when doing discovery research we can start with interviews and once we have identified some themes, use a quantitative method to validate our insights. This process can be reversed as well depending on the research question we have.
Limitations of triangulation
Triangulation in whatever form is based on the assumption that using several data sources, methods, or researchers will reduce any bias in a data set or methodological approach. This means that using triangulation allows us to increase the confidence we have in our insights. This approach, however, has a caveat; it increases the chances of confirmation bias. If you’re not familiar with it, Confirmation bias “describes our underlying tendency to notice, focus on, and give greater credence to evidence that fits with our existing beliefs”.
If you haven’t used triangulation before you should keep in mind that it can sometimes result in contradictory and inconsistent results (Fusch et al., 2018). In these cases, it is up to the researcher to make sense of the data and understand the source of the inconsistency. This is something particularly challenging for less experienced researchers.
Triangulation can help us acquire a fuller picture but not a complete picture of the phenomenon we’re researching. It goes without saying that proper study design is still crucial to ensure credibility. Using triangulation isn’t going to help you if your study design is poor — the quality of output is determined by the quality of input (Garbage in Garbage Out concept).
When should you triangulate?
In an ideal world, we’d have months and an unlimited budget to conduct research. In reality, we often find ourselves with barely enough time to use one method as rigidly as we would like and with limited resources. This makes using triangulation quite challenging and not something we can do for every study. So how do we decide when to triangulate?
Triangulate when you’re doing work of high importance: We can’t always triangulate but we should prioritise it when making important decisions that could affect the business and the users (e.g., major redesign).
If you’re lucky enough to have multiple researchers in your team, work as a group and involve at least 2 researchers in the data collection and the analysis. This will allow you to control for a number of biases (e.g., interviewer bias).
Collaborate with other teams: when working on a new project you can start by looking at existing data and product analytics. Involve the data analysts and the Product managers.
How often do you use triangulation in your research? Use this link to vote or leave a comment below.
Thank you María for sharing. It has helped me a lot to know when to apply different techniques. 🙌🏻✨
Great article Dr. Panagiotidi