The Creator's Blind Spot

Unveiling the Inventor's Bias Effect in Product Development

As our world increasingly relies on artificial intelligence (AI), understanding the psychology of those who create these tools becomes more and more critical. I recently came across an interesting new study by Cratsley and Fast that explores the "Inventor's Bias Effect", a phenomenon where inventors hold an overly optimistic view of their own creations' features and applications. This research, published in the International Journal of Human-Computer Interaction, offers some insight into decision-making biases associated with technology development.

The Study

When we imagine the mind of an inventor, one might suppose they would evaluate their own products critically. After all, “everyone is their own worst critic”! Yet, Cratsley and Fast’s research challenges this notion and unravels a complex picture where identity, self-esteem, and self-concept play significant roles in how creators perceive their products.

The study consisted of two online experiments involving 1,001 participants on decision-making algorithms. The participants were tasked with either developing or evaluating an AI tool designed to automate human resource decisions within an organisation.

In the first experiment, inventors were observed to view the low-performing algorithm they developed as fairer compared to the perception of other stakeholders, such as CEOs, employees, and the general public. The personal identification of inventors with their creations emerged as a significant contributing factor to this effect.

The second experiment further illuminated the inventors' inclination to continue using their own algorithms within the organisation, even when their product was found to be inaccurate in one-third of all decisions. This tendency was motivated by the inventors' perceived fairness of their own algorithms.

This set of discoveries underscores how stakeholders' relationship with algorithms can foster biased decision-making, thus necessitating caution in organisational decision-making processes (and beyond).

The Inventor’s Bias Effect

Building on these insights, the researchers theorise that the “Inventor’s Bias Effect” arises from inventors incorporating the products they create into their self-identity. This idea is grounded on two main considerations.

Firstly, when an individual creates something, such as an algorithmic tool or even a product, their choices are informed by their own preferences and beliefs, which are integral components of their identity. This makes their invention an extension of themselves.

Secondly, creating something typically involves substantial time, effort, and resources. Given this personal investment, inventors may perceive their creations as an important self-expression. These two considerations collectively explain why inventors might view their creations more positively, hence the “Inventor’s Bias Effect”.

In the context of AI developers who were the focus of this study, this means an over-perception of fairness in the AI tools they develop, compared to other stakeholders' perceptions. This finding is particularly relevant considering that in the technology sector, it is often the creators themselves who are regarded as the ultimate authorities on their creations.

Implications for Users and the General Public

Cratsley and Fast also make the case that this phenomenon is not applicable to other stakeholders, such as users, subjects, and members of the general public. While inventors may identify with their creations, these other stakeholders typically don't share this tendency.

The often "black box" nature of many algorithms also means that users, subjects, and the general public, who are not involved in the creation of the algorithms, have little understanding of how the algorithms make their predictions. This lack of knowledge and involvement makes it less likely for them to identify with the algorithm or view it as an expression of themselves.

This research paints a complex picture of the psychology behind the creators of AI tools, shining a light on the inherent biases these creators may have towards their inventions. As we continue to adopt AI into more aspects of our lives, this understanding becomes vital. It highlights the need to maintain a level of caution regarding creators’ evaluations of their products, particularly in organisational and political decision-making processes.

What this ultimately uncovers is that the Inventor's Bias Effect can have far-reaching implications on decisions about when and where to use AI tools. For instance, relying too heavily on developers' evaluations due to their expert status can lead to overblown expectations regarding the fairness and efficiency of the algorithms. This is particularly concerning given the "black box" nature of many AI tools, which may encourage an increased dependence on the creators of AI as expert evaluators of their own tools.

This research also underscores the importance of a more diverse input in the decision-making process. As different stakeholders may have varying degrees of identification with an algorithm, they will also likely have diverse perceptions of its fairness and performance.

In the realm of technology regulation, these findings raise important questions around the concept of self-governance. While many tech leaders argue for self-regulation, arguing that they are best equipped to oversee their own products, the Inventor’s Bias Effect suggests that they may also be prone to favour their own technologies. This brings to the fore the question of whether they can truly govern effectively and objectively.

Implications for UX - “You are not the user”

The "Inventor's Bias Effect" closely aligns with the well-known UX mantra: "You are not the user." This principle prompts us to be aware of our potential biases and to make design decisions based on user data rather than our personal preferences or beliefs.

The concept of inventors being overly optimistic about their own creations is in line with this. People involved in Product Development, similar to the inventors in the study, often have an intimate relationship with the products they create and may identify with them to a great extent. This could potentially lead to an overestimation of the usability and effectiveness of their solutions and an underestimation of the potential issues and challenges users might face.

How can we overcome this bias?

Conduct UX research with our users. Once more, research comes to the rescue! By doing so, we can prevent the "Inventor's Bias Effect" from distorting our perceptions of our designs, and instead create products that truly meet users' needs.

In conclusion, as the world becomes more reliant on AI and technology, understanding the psychology of those who create these tools is paramount. This latest research on the "Inventor's Bias Effect" provides useful insights in this regard. It serves as a reminder for UX professionals, tech leaders, and decision-makers to remain vigilant about their potential biases and to continually strive for objectivity and user-centricity in their work…

---

A Favour

I’m conducting a short study to better understand how UX professionals use AI in their work and how they feel about it. If you have a few minutes to spare, can you help me out by completing the survey or sharing it with your network? The results will be shared with the community!