The Role of Personality and Cognitive Ability in the Measurement of Usability
The Role of Personality and Cognitive Ability in the Measurement of Usability
Implications for UX Practice
The field of User Experience (UX) continues to evolve, with ongoing research contributing to our understanding of how users interact with digital interfaces. A recent study, published in the Journal of Usability Studies, provides interesting insights into the role of personality traits and cognitive ability in the subjective assessment of usability. This post will delve into the key findings of this research and the implications for UX.
The usability of a product isn't a fixed attribute, but rather, a spectrum that varies among individuals based on their knowledge, experience, and attitudes towards the product. Therefore, one of the pressing questions in usability research is understanding who finds a product more or less usable than the average usability characterising the product.
A widely recognised instrument for measuring subjective usability is the System Usability Scale (SUS), which has gained acceptance due to its reliability, validity, and ease of interpretation. I have previously wrote about it in my Standardized usability questionnaires: which one to use? post. Using the SUS, we can calculate average usability scores for products. Yet, it's equally important to explore how individual characteristics influence usability ratings for a given product.
Traditional demographic variables, such as age and gender, have often been considered to have little to no impact on SUS scores. However, recent research, including a study by Kortum and Oswald (2018), suggests that personality traits such as openness to experience and agreeableness correlate positively with usability ratings.
New research
A recent study by Wilczewski and colleagues published in the Journal of Usability Studies investigated the role of personality and cognitive ability in the subjective measurement of the usability of 7 products: Microsoft Word, Google Search, Amazon, personal computer, Microsoft Excel, TV remote, and microwave.
The researchers examined the relationship between personality, cognitive ability, and usability scores. The study included 972 participants recruited through the survey platform Prolific. The sample was 54% female and 46% male, with an age range of 18-81 years and a mean age of 43. This provided a diverse group to examine individual differences.
Personality was assessed using the Mini-IPIP scale. This validated questionnaire measures the Big Five traits of extraversion, agreeableness, conscientiousness, emotional stability (neuroticism), and openness to experience. It consists of 20 items rated on a 5-point scale. Each trait is scored by summing the relevant items.
A brief definition of each of the 5 personality traits is provided below:
Openness - Appreciation for new experiences, ideas, and creativity. People high in openness tend to be curious and open-minded.
Conscientiousness - Tendency to be organised, responsible, and hardworking. Highly conscientious people are efficient and achievement-focused.
Extraversion - Orientation towards engaging with external stimuli. Extraverts tend to be outgoing, energetic, and seek social interaction.
Agreeableness - Tendency to be compassionate, cooperative, and trusting. Highly agreeable people value harmony and are considerate of others.
Neuroticism (Emotional stability) - Propensity to experience emotional instability and negative emotions like anxiety or anger. Neurotic people are more likely to experience stress and be self-conscious.
Cognitive ability refers to the general mental capability that influences how well an individual can learn, solve problems, and comprehend complex ideas. It includes faculties such as memory, reasoning, attention, language. In this study it was evaluated by using a matrix reasoning test requiring pattern recognition. Participants completed 20 multiple-choice questions without the option to go back. Performance was scored as the number of correct responses, with a maximum of 20.
The widely-used System Usability Scale (SUS) was selected to assess perceived usability. Participants rated 7 products on a 5-point Likert scale across 10 items alternating between positive and negative wording. Products were presented randomly. SUS is scored from 0-100, with 100 being very high usability.
Key Findings
Surprisingly, cognitive ability did not significantly impact the subjective usability assessment, contrary to common beliefs. Instead, small-to-moderate predictors of System Usability Scale (SUS) scores were identified in individual personality traits, explaining between 6–12% of the variance in usability. This aligns with previous findings that indicated individual differences can significantly impact usability assessment.
Personality Traits as Predictors
The research identified that personality traits like emotional stability, openness to experience, and conscientiousness consistently dictated higher subjective usability ratings across the tested products. This section delves into the insights derived from these observations:
Emotional Stability: Individuals with higher emotional stability demonstrated greater patience during product usage, potentially experiencing a more positive usability assessment compared to those with lower emotional stability.
Openness to Experience: This trait predicted higher usability ratings across all products, as people with higher openness levels displayed a more curious and exploratory nature, keen to delve into product features comprehensively.
Conscientiousness: This trait influenced higher ratings for five out of seven products, indicating a preference for products mirroring these traits in individuals high in conscientiousness.
The data strongly implies that personality significantly molds the subjective perceptions of usability, steering individuals to view product usability more positively.
Cognitive Ability: Not a Significant Predictor
Contrary to personality traits, cognitive ability did not significantly influence usability ratings for most products when other factors were controlled. This suggests a limited role of general mental capabilities in shaping usability perceptions for mainstream consumer products, except potentially for complex interfaces demanding intense mental engagement.
Demographic Factors: A Subtle Influence
While gender and age had a minimal impact on usability assessments, they showcased mixed results. Generally, women offered slightly higher usability scores, and younger individuals tended to rate products slightly more favourably, although these effects were small and inconsistent.
Implications for UX Professionals
One key implication from this research is that these constructs of personality should be understood when administering usability assessments. This understanding can allow us to not only evaluate the average level of usability for a given system but also consider the variability in reported usability, which can be influenced by individual differences. Here are a few things we can do to account for the influence of personality on subjective usability assessments:
Recruit test participants with diverse personalities: Don't rely solely on users who may rate usability higher due to certain traits. Seek out both high and low scorers on key dimensions like emotional stability, openness, and conscientiousness. Recruiting users from diverse backgrounds and using various incentives can help with this.
Consider personality composition when interpreting results: If the test sample skews towards certain traits, acknowledge this in reporting and how it may have impacted ratings. Don't overgeneralise findings if the sample is too limited.
Use multiple methods beyond self-report measures: Include observational and behavioural usability metrics to complement rating scale data, which can be biased by personality factors.
Conduct testing across different stages: Personality may play a bigger role in early stage evaluations, or when the functionality of the prototype/product is limited, versus later stage testing with more refined prototypes.
I'd love to hear from you – do you employ any other strategies in your subjective usability assessments? Feel free to share your thoughts in the comments below!