Diversity & AI
Diversity & AI
The research line on Diversity & AI investigates from an intersectional approach how robots, AI, and algorithmic decision-making processes impact different communities, including the LGBTQAI+, the elderly, or under-resourced communities.
Discrimination and bias are inherent problems of many AI applications, as seen in, for instance, face recognition systems not recognizing dark-skinned women and content moderator tools silencing drag queens online. These outcomes may derive from limited datasets that do not fully represent society as a whole or from the AI scientific community's western-male configuration bias. Although being a pressing issue, understanding how AI systems can replicate and amplify inequalities and injustice among underrepresented communities is still in its infancy in social science and technical communities. This chapter contributes to filling this gap by exploring the research question: what do diversity and inclusion mean in the context of AI? This chapter reviews the literature on diversity and inclusion in AI to unearth the underpinnings of the topic and identify key concepts, research gaps, and evidence sources to inform practice and policymaking in this area. Here, attention is directed to three different levels of the AI development process: the technical, the community, and the target user level. The latter is expanded upon, providing concrete examples of usually overlooked communities in the development of AI, such as women, the LGBTQ+ community, senior citizens, and disabled persons. Sex and gender diversity considerations emerge as the most at risk in AI applications and practices and thus are the focus here. To help mitigate the risks that missing sex and gender considerations in AI could pose for society, this chapter closes with proposing gendering algorithms, more diverse design teams, and more inclusive and explicit guiding policies. Overall, this chapter argues that by integrating diversity and inclusion considerations, AI systems can be created to be more attuned to all-inclusive societal needs, respect fundamental rights, and represent contemporary values in modern societies.
Fosch-Villaronga, E., Drukarch, H, Khanna, P., Verhoef, T., & Custers, B. (2022). Accounting for diversity in AI for medicine. Computer Law and Security Review, 47, 105735.
In healthcare, gender and sex considerations are crucial because they affect individuals' health and disease differences. Yet, most algorithms deployed in the healthcare context do not consider these aspects and do not account for bias detection. Missing these dimensions in algorithms used in medicine is a huge point of concern, as neglecting these aspects will inevitably produce far from optimal results and generate errors that may lead to misdiagnosis and potential discrimination. This paper explores how current algorithmic-based systems may reinforce gender biases and affect marginalized communities in healthcare-related applications. To do so, we bring together notions and reflections from computer science, queer media studies, and legal insights to better understand the magnitude of failing to consider gender and sex difference in the use of algorithms for medical purposes. Our goal is to illustrate the potential impact that algorithmic bias may have on inadvertent discriminatory, safety, and privacy-related concerns for patients in increasingly automated medicine. This is necessary because by rushing the deployment of AI technologies that do not account for diversity, we risk having an even more unsafe and inadequate healthcare delivery. By promoting the account for privacy, safety, diversity, and inclusion in algorithmic developments with health-related outcomes, we ultimately aim to inform the Artificial Intelligence (AI) global governance landscape and practice on the importance of integrating gender and sex considerations in the development of algorithms to avoid exacerbating existing or new prejudices.
Calleja, C., Drukarch, H., & Fosch-Villaronga, E. (2022). Diversity observations in an exoskeleton experiment. Inclusive HRI Workshop: Equity and Diversity in Design, Application, Methods, and Community, March 11, 2022,.
While robots in medical care are becoming increasingly prevalent, direct interaction with users raises new ethical and social issues that have an impact on the law and regulatory initiatives. One of those concerns, still underexplored, is how to make these robots fit for users that come in different shapes, sizes, and genders. Although mentioned in the literature, these concerns have not yet been reflected in industry standards. For instance, ISO 13482:2014 on safety requirements for personal care robots briefly acknowledges that future editions might include more information about different kinds of people. More than seven years following its approval and after undergoing revision, those requirements are nonetheless still missing. Based on the tests with robotic exoskeletons as part of the H2020 EUROBENCH FSTP PROPELLING, we argue that being oblivious to differences in gender and medical conditions or following a one-size-fits-all approach hides important distinctions and increases the exclusion of specific users. Our observations show that robotic exoskeletons operate intimately with users' bodies, thus exemplifying how gender and medical conditions might introduce dissimilarities in human-robot interaction that, as long as they remain ignored in regulations, may compromise user safety. We conclude the article by putting forward particular recommendations to update ISO 13482:2014 to reflect better the broad diversity of users of personal care robots.
Singh, D. K., Kumar, M., Fosch-Villaronga, E., Singh, D., & Shukla, J. (2022). Ethical Considerations from Child-Robot Interactions in Under-Resourced Communities. International Journal of Social Robotics, 1-17.
Recent advancements in socially assistive robotics (SAR) have shown a significant potential of using social robotics to achieve increasing cognitive and affective outcomes in education. However, the deployments of SAR technologies also bring ethical challenges in tandem, to the fore, especially in under-resourced contexts. While previous research has highlighted various ethical challenges that arise in SAR deployment in real-world settings, most of the research has been centered in resource-rich contexts, mainly in developed countries in the ‘Global North,’ and the work specifically in the educational setting is limited. This research aims to evaluate and reflect upon the potential ethical and pedagogical challenges of deploying a social robot in an under-resourced context. We base our findings on a 5-week in-the-wild user study conducted with 12 kindergarten students at an under-resourced community school in New Delhi, India. We used interaction analysis with the context of learning, education, and ethics to analyze the user study through video recordings. Our findings highlighted four primary ethical considerations that should be taken into account while deploying social robotics technologies in educational settings; (1) language and accent as barriers in pedagogy, (2) effect of malfunctioning, (un)intended harms, (3) trust and deception, and (4) ecological viability of innovation. Overall, our paper argues for assessing the ethical and pedagogical constraints and bridging the gap between non-existent literature from such a context to evaluate better the potential use of such technologies in under-resourced contexts.
Fosch-Villaronga, E., Poulsen, A., Søraa, R. A., & Custers, B. H. M. (2020) Gendering Algorithms in Social Media. ACM SIGKDD Explorations Newsletter Volume 23 Issue 1, June 2021, 24–31.
Social media platforms employ inferential analytics methods to guess user preferences and may include sensitive attributes such as race, gender, sexual orientation, and political opinions. These methods are often opaque, but they can have significant effects such as predicting behaviors for marketing purposes, influencing behavior for profit, serving attention economics, and reinforcing existing biases such as gender stereotyping. Although two international human rights treaties include express obligations relating to harmful and wrongful stereotyping, these stereotypes persist both online and offline, and platforms often appear to fail to understand that gender is not merely a binary of being a 'man' or a 'woman,' but is socially constructed. Our study investigates the impact of algorithmic bias on inadvertent privacy violations and the reinforcement of social prejudices of gender and sexuality through a multidisciplinary perspective including legal, computer science, and queer media viewpoints. We conducted an online survey to understand whether and how Twitter inferred the gender of users. Beyond Twitter's binary understanding of gender and the inevitability of the gender inference as part of Twitter's personalization trade-off, the results show that Twitter misgendered users in nearly 20% of the cases (N=109). Although not apparently correlated, only 8% of the straight male respondents were misgendered, compared to 25% of gay men and 16% of straight women. Our contribution shows how the lack of attention to gender in gender classifiers exacerbates existing biases and affects marginalized communities. With our paper, we hope to promote the online account for privacy, diversity, and inclusion and advocate for the freedom of identity that everyone should have online and offline. Also available on ResearchGate.
Gender inferences in social media
Fosch-Villaronga, E., Poulsen, A., Søraa, R. A., & Custers, B. H. M. (2020) A little bird told me your gender: Gender inferences in social media. Information Processing and Management, 58(3), 102541.
Poulsen, A., Fosch-Villaronga, E., Burmeister, O. K. (2020) Cybersecurity, value sensing robots for LGBTIQ+ elderly, and the need for revised codes of conduct. Australasian Journal of Information Systems 24, 1-16.
Until now, each profession has developed their professional codes of conduct independently. However, the use of robots and artificial intelligence is blurring professional delineations: aged care nurses work with lifting robots, tablet computers, and intelligent diagnostic systems, and health information system designers work with clinical teams. While robots assist the medical staff in extending the professional service they provide, it is not clear how professions adhere and adapt to the new reality. In this article, we reflect on how the insertion of robots may shape codes of conduct, in particular with regards to cybersecurity. We do so by focusing on the use of social robots for helping LGBTIQ+ elderly cope with loneliness and depression. Using robots in such a delicate domain of application changes how care is delivered, as now alongside the caregiver, there is a cyber-physical health information system that can learn from experience and act autonomously. Our contribution stresses the importance of including cybersecurity considerations in codes of conduct for both robot developers and caregivers as it is the human and not the machine which is responsible for ensuring the system’s security and the user’s safety.
Søraa, R. A. & Fosch-Villaronga, E., (2020) Exoskeletons for all: The interplay between exoskeletons, inclusion, gender and intersectionality. Paladyn Journal of Behavioral Robotics 11(1), 217-227
In this article, we investigate the relation between gender and exoskeleton development through the lens of intersectionality theory. Exoskeleton users come in a wide variety of shapes, sizes, and genders. However, it is often the case that wearable robot engineers do not develop such devices primarily on the premise that the product should fit as many end users as possible. Instead, designers tend to use the one-size-fits-all approach – a design choice that seems legitimate from the return of an investment viewpoint but that may not do as much justice to end users. Intended users of exoskeletons have a series of user criteria, including height, weight, and health condition, in the case of rehabilitation. By having rigid inclusion criteria for whom the intended user of the technology can be, the exclusion criteria will grow in parallel. The implications and deep-rootedness of gender and diversity considerations in practices and structural systems have been largely disregarded. Mechanical and robot technology were historically seen as part of a distinct male sphere, and the criteria used today to develop new technology may reflect the biases that existed in another time that should no longer be valid. To make this technology available for all, we suggest some tools to designers and manufacturers to help them think beyond their target market and be more inclusive.