Designing for Autonomy: Balancing User Control and AI Assistance in Recommender Systems
New Research Insights into Recommender System Design
As artificial intelligence (AI) becomes increasingly integrated into our daily lives and decision-making processes, concerns have arisen about its potential to undermine human autonomy. Recommender systems, which use AI to suggest personalised content and products to users, are at the forefront of this issue. While these systems can improve users’ decision making, they can also leave them feeling coerced or manipulated (André et al., 2018). This has been linked to algorithm aversion - a reluctance to rely on AI advice even when it outperforms human judgment (Dietvorst et al., 2015) - and algorithm appreciation, an over-reliance on AI output (Logg et al., 2019).
At this point, it is crucial to understand how to balance the benefits of AI assistance with respect for user autonomy and apply the knowledge to the design of recommender systems. A recent systematic review by Burton et al. (2020) suggested that lack of autonomy is a key driver of algorithm aversion, suggesting that increasing user control could improve acceptance of AI recommendations. This aligns with long-standing psychological research on the importance of autonomy for motivation and well-being.
Self-Determination Theory and Autonomy
Self-determination theory, developed by Ryan and Deci (2000), is a macro theory of human motivation that emphasises the importance of three basic psychological needs: autonomy, competence, and relatedness. According to this theory, environments that support these needs foster intrinsic motivation, leading to enhanced performance, persistence, and creativity.
Autonomy is defined as the need to self-regulate one's experiences and actions and is considered the most central of the three needs. When people have high autonomy, they experience their behaviour as self-endorsed and congruent with their values and interests. In contrast, feeling controlled or pressured negatively affects intrinsic motivation and well-being. Many studies have shown the benefits of autonomy-supportive environments in various domains such as workplace and education.
Applied to the context of AI-assisted decision-making, self-determination theory suggests that supporting user autonomy could increase user engagement with AI recommendations. When users feel that they have meaningful choices and control over their interactions with a recommender system, they may be more likely to view the system as a helpful tool rather than a coercive influence. However, few studies have examined specific design mechanisms for promoting autonomy in recommender systems.
Cognitive Biases and Algorithm Aversion
In addition to psychological needs, cognitive biases may also contribute to algorithm aversion. One relevant bias is the illusion of control, which refers to people's tendency to overestimate their ability to control outcomes (Langer, 1975). In the context of decision-making, this bias can lead people to ignore advice from external sources, including AI, in favour of their own judgment. Supporting user autonomy may help counteract this bias by giving users a sense of control, even if the actual impact of their choices is limited.
Another bias that can lead to algorithm aversion is the anchoring effect, which occurs when people rely too heavily on an initial piece of information when making decisions (Tversky & Kahneman, 1974). In the case of recommender systems, a single recommendation could serve as an anchor, biasing users' evaluations of subsequent options. Presenting multiple recommendations simultaneously could mitigate this effect by providing users with a more diverse set of initial information.
Using Design to Promote User Autonomy
A recent study by Fink, Newman, and Haran (2024) sought to examine how specific design choices in recommender systems could support user autonomy and influence acceptance of AI recommendations. The researchers conducted three online experiments using a simulated vacation package recommendation system.
In each experiment, participants first inputted their preferences for various vacation options, such as location, activities, and accommodation type. The system then presented participants with either a single recommendation (low choice autonomy condition), three recommendations (high choice autonomy condition), or an initial single recommendation with the option to reveal two more (high control autonomy condition).
In the first experiment, the system always recommended the objectively best option based on participants' stated preferences. In the second one, the recommendations were deliberately suboptimal, with the best option withheld. This allowed the researchers to disentangle the effects of recommendation quality and autonomy on acceptance rates. The final experiment introduced the control autonomy condition, in which participants could choose whether to receive one or three recommendations.
Across all experiments, the dependent variable was recommendation acceptance —whether participants selected an option that the system had recommended. The researchers also collected data on decision time, completion time, and various demographic and contextual variables.
The results of the three experiments consistently supported the hypothesis that increased autonomy would lead to higher recommendation acceptance. In the first experiment, participants in the high choice autonomy condition (three recommendations) were significantly more likely to select a recommended option compared to those in the low choice autonomy condition (single recommendation). This effect was replicated in the second experiment, even though the recommendations were no longer the best available options.
There were two key findings from the third experiment. First, high choice autonomy once again increased recommendation acceptance, both when the number of recommendations was fixed and when it was under participants' control. Second, high control autonomy (the ability to choose between one and three recommendations) moderated the effect of choice autonomy. Participants who had both choice and control autonomy showed the highest acceptance rates, while those with neither showed the lowest.
Importantly, the benefits of increased autonomy did not come at the cost of decision quality. In the second experiment, participants in the high choice autonomy condition were just as likely to select the best available option as those in the low choice autonomy condition, despite receiving objectively worse recommendations. This suggests that the positive effects of autonomy can outweigh minor reductions in recommendation quality.
Implications and Future Directions
The findings of Fink et al. (2024) have important implications for the design of recommender systems and other AI-assisted decision-making tools. At a basic level, they demonstrate that simple interface changes, such as presenting multiple recommendations instead of a single one, can significantly increase user acceptance of AI advice. This effect appears to be driven by increased perceptions of choice autonomy, consistent with the predictions of self-determination theory.
More broadly, the study highlights the importance of designing for user autonomy in AI systems. By providing users with meaningful choices and control over their interactions with AI, designers can foster a sense of self-endorsement and intrinsic motivation. This, in turn, may help counteract algorithm aversion and improve user trust and engagement.
The study also raises intriguing questions about the interplay between different forms of autonomy support. The finding that choice and control autonomy had a synergistic effect suggests that multiple autonomy-supportive features may be more effective than any single feature alone. Future research could explore other ways of combining different types of autonomy support, such as allowing users to adjust the parameters of the recommendation algorithm or providing explanations for why certain options were recommended.
Another important direction for future work is to examine boundary conditions and contextual factors that may influence the effects of autonomy support. For example, the complexity of the decision domain, the stakes involved, and individual differences in technology literacy or decision-making style could all moderate the impact of autonomy-supportive features. Longitudinal studies could also investigate whether the benefits of autonomy support persist over time or if users become habituated to certain features.
Finally, the ethical implications of autonomy support in AI systems deserve further attention. While increased autonomy is generally seen as a positive, there may be cases where too much choice or control could be detrimental, such as when users lack the necessary knowledge or motivation to make informed decisions. Designers will need to strike a careful balance between supporting autonomy and providing appropriate guidance and structure.
Recommendations for UX Professionals
According to the research findings the following recommendations can improve the way we design recommender systems:
Provide multiple recommendations: Instead of presenting users with a single "best" recommendation, consider offering a small set of high-quality options. This allows users to exercise choice while still benefiting from the AI's curation. Aim for a "Goldilocks" number of recommendations - not so few that users feel constrained, but not so many that they become overwhelmed. Conduct user research to identify what that number is.
Give users control over the recommendation process: Look for opportunities to let users customise and adjust the AI to their needs. For example, allowing users to specify the number of recommendations they want to see or the relative importance of different criteria can enhance their sense of control. Make these controls easily accessible and clearly explain their effects.
Combine different types of autonomy support: The study suggests that multiple autonomy-supportive features may have synergistic effects. Consider how you can provide both choice and control within the same system. For example, you might allow users to choose from a set of recommended options and also adjust the parameters that generated those recommendations.
Make the AI's reasoning transparent: To help users make informed decisions, provide explanations for why certain options were recommended. Use clear, concise language and avoid technical jargon. Highlight the key factors that influenced the AI's suggestions and how they relate to the user's stated preferences.
Allow users to give feedback: Provide mechanisms for users to rate, comment on, or otherwise react to the AI's recommendations. This not only enhances their sense of autonomy but also provides valuable data for improving the system over time. Communicate how user feedback is being used and its impact on future recommendations.
Test and iterate: As with any UX design, it's crucial to test autonomy-supportive features with real users and gather feedback! Secondary research does not replace user research! Pay attention to both objective metrics (e.g., recommendation acceptance rates) and subjective experiences (e.g., perceived autonomy, trust, and satisfaction). Use this data to continually refine and improve the system.
Consider the context: The appropriate level and type of autonomy support may vary depending on the domain, user characteristics, and stage of the user journey. For high-stakes decisions or novice users, more structured guidance may be needed. For low-stakes, repetitive choices or expert users, greater autonomy may be preferred. Adapt your design to the specific needs of your users and use case.
Balance autonomy with other design (and business) goals: While autonomy is important, it's not the only factor influencing user engagement and satisfaction. Consider how autonomy-supportive features interact with other design elements such as usability, aesthetics, and efficiency. Strive for a harmonious balance that meets users' overall needs and preferences.
Do you have any other tips or suggestions on how to design more effective recommendation systems? I'd love to hear your thoughts and experiences in the comments below.