Reflections on robots & autonomous systems

Exploring the consequences of risk assessment automation

Fosch-Villaronga, E., Drukarch, H., and Callegaro, G.

Exploring the consequences of risk assessment automation.pdf

Büchi, M., Fosch-Villaronga, E., Lutz, C., Tamò-Larrieux, A., & Velidi, S. (2021). Making sense of algorithmic profiling: user perceptions on Facebook. Information, Communication & Society, 1-17.

Algorithmic profiling has become increasingly prevalent in many social fields and practices, including finance, marketing, law, cultural consumption and production, and social engagement. Although researchers have begun to investigate algorithmic profiling from various perspectives, socio-technical studies of algorithmic profiling that consider users’ everyday perceptions are still scarce. In this article, we expand upon existing user-centered research and focus on people’s awareness and imaginaries of algorithmic profiling, specifically in the context of social media and targeted advertising. We conducted an online survey geared toward understanding how Facebook users react to and make sense of algorithmic profiling when it is made visible. The methodology relied on qualitative accounts as well as quantitative data from 292 Facebook users in the United States and their reactions to their algorithmically inferred ‘Your Interests’ and ‘Your Categories’ sections on Facebook. The results illustrate a broad set of reactions and rationales to Facebook’s (public-facing) algorithmic profiling, ranging from shock and surprise, to accounts of how superficial – and in some cases, inaccurate – the profiles were. Taken together with the increasing reliance on Facebook as critical social infrastructure, our study highlights a sense of algorithmic disillusionment requiring further research.

Putting children and their rights at the forefront of the artificial intelligence revolution

Fosch-Villaronga, E., van der Hof, S., Lutz, C., & Tamò-Larrieux, A. (2021). Toy story or children story? Putting children and their rights at the forefront of the artificial intelligence revolution. Ai & Society, 1-20.

Policymakers need to start considering the impact smart connected toys (SCTs) have on children. Equipped with sensors, data processing capacities, and connectivity, SCTs targeting children increasingly penetrate pervasively personal environments. The network of SCTs forms the Internet of Toys (IoToys) and often increases children's engagement and playtime experience. Unfortunately, this young part of the population and, most of the time, their parents are often unaware of SCTs’ far-reaching capacities and limitations. The capabilities and constraints of SCTs create severe side effects at the technical, individual, and societal level. These side effects are often unforeseeable and unexpected. They arise from the technology's use and the interconnected nature of the IoToys, without necessarily involving malevolence from their creators. Although existing regulations and new ethical guidelines for artificial intelligence provide remedies to address some of the side effects, policymakers did not develop these redress mechanisms having children and SCTs in mind. This article provides an analysis of the arising side effects of SCTs and contrasts them with current regulatory redress mechanisms. We thereby highlight misfits and needs for further policymaking efforts.

Søraa, R. A., Nyvoll, P., Tøndel, G., Fosch-Villaronga, E., & Serrano, J. A. (2021). The social dimension of domesticating technology: Interactions between older adults, caregivers, and robots in the home. Technological Forecasting and Social Change, 167, 120678.

In this article we look at the home as an arena for care by exploring how care robots and technological care-systems can become part of older adults’ lives. We investigate the domestication of robot technology in the context of what in Scandinavia is called “welfare technology” (relating to the terms “gerontechnology” and “Active Assisted Living,”) that especially aims to mitigate older adults´ challenges with living in their own homes. Through our case study, we investigate a system called eWare, where a flowerpot robot called “Tessa” works in symbiosis with a sensor technology “SensaraCare.” Together, they create a socio-technical ecosystem involving older adult end-users living at home, formal caregivers (e.g. healthcare workers), and informal caregivers (normally family members). We analyze our ethnographic fieldwork through the theoretical concept of “domestication of technology,” focusing on an established three-dimensional model that includes practical, symbolic, and cognitive levels of analysis. We found that social bonds and different ways of using the same technology ecosystem were crucial, and so we supplement this model by suggesting a fourth dimension, which we term the social dimension of the domestication of technology.


Fosch-Villaronga, E., Lutz, C., Tamò-Larrieux, A. (2020) Gathering expert opinions for social robots' ethical, legal, and societal concerns. Findings from Four International Workshops. International Journal of Social Robotics, 12(4), 959-972.

Social robots, those that exhibit personality and communicate with us using high-level dialogue and natural cues, will soon be part of our daily lives. In this paper, we gather expert opinions from different international workshops exploring ethical, legal, and social (ELS) concerns associated with social robots. In contrast to literature that looks at specific challenges, often from a certain disciplinary angle, our contribution to the literature provides an overview of the ELS discussions in a holistic fashion, shaped by active deliberation with a multitude of experts across four workshops held between 2015 and 2017 held in major international workshops (ERF, NewFriends, JSAI-isAI). It also explores pathways to address the identified challenges. Our contribution is in line with the latest European robot regulatory initiatives but covers an area of research that the latest AI and robot governance strategies have scarcely covered. Specifically, we highlight challenges to the use of social robots from a user perspective, including issues such as privacy, autonomy, and the dehumanization of interactions; or from a worker perspective, including issues such as the possible replacement of jobs through robots. The paper also compiles the recommendations to these ELS issues the experts deem appropriate to mitigate compounding risks. By then contrasting these challenges and solutions with recent AI and robot regulatory strategies, we hope to inform the policy debate and set the scene for further research.

Buechi, M., Fosch-Villaronga, E., Lutz, C., Tamò, A., Velidi, S., and Viljoe, S. (2020) The Chilling Effects of Algorithmic Profiling: Mapping the Issues. Computer Law & Security Law Review, 36, 105367,1-15.

In this article, we provide an overview of the literature on chilling effects and corporate profiling, while also connecting the two topics. We start by explaining how profiling, in an increasingly data-rich environment, creates substantial power asymmetries between users and platforms (and corporations more broadly). Inferences and the increasingly automated nature of decision-making, both based on user data, are essential aspects of profiling. We then connect chilling effects theory and the relevant empirical findings to corporate profiling. In this article, we first stress the relationship and similarities between profiling and surveillance. Second, we describe chilling effects as a result of state and peer surveillance, specifically. We then show the interrelatedness of corporate and state profiling, and finally spotlight the customization of behavior and behavioral manipulation as particularly significant issues in this discourse. This is complemented with an exploration of the legal foundations of profiling through an analysis of European and US data protection law. We find that while Europe has a clear regulatory framework in place for profiling, the US primarily relies on a patchwork of sector-specific or state laws. Further, there is an attempt to regulate differential impacts of profiling via anti-discrimination statutes, yet few policies focus on combating generalized harms of profiling, such as chilling effects. Finally, we devise four concise propositions to guide future research on the connection between corporate profiling and chilling effects.

Zardiashvili, L. & Fosch-Villaronga, E. (2020) ‘Oh, dignity too?’ said the robot. Human Dignity as the Basis of the Governance of Robotics. Minds and Machines, 30, 121–143.

Healthcare robots enable practices that seemed far-fetched in the past. Robots might be the solution to bridge the loneliness that the elderly often experience; they may help wheelchair users walk again, or may help navigate the blind. European Institutions, however, acknowledge that human contact is an essential aspect of personal care and that the insertion of robots could dehumanize caring practices. Such instances of human–robot interactions raise the question to what extent the use and development of robots for healthcare applications can challenge the dignity of users. In this article, therefore, we explore how different robot applications in the healthcare domain support individuals in achieving ‘dignity’ or pressure it. We argue that since healthcare robot applications are novel, their associated risks and impacts may be unprecedented and unknown, thus triggering the need for a conceptual instrument that is binding and remains flexible at the same time. In this respect, as safety rules and data protection are often criticized to lack flexibility, and technology ethics to lack enforceability, we suggest human dignity as the overarching governance instrument for robotics, which is the inviolable value upon which all fundamental rights are grounded

Fosch-Villaronga, E. (2019) “I love you,” said the robot. Boundaries of the use of emotions in human-robot interaction. In Ayanoglu, H., and Duarte, E. (eds.) (2019) Emotional Design in Human Robot Interaction: Theory, Methods, and Application. Human-Computer Interaction Series, Springer, 93-110.

This chapter reflects upon the ethical, legal, and societal (ELS) implications of the use of emotions by robot technology. The first section introduces different cases where emotions play a role in human-robot interaction (HRI) contexts. This chapter draws particular attention to disparities found in recent technical literature relating to the appropriateness of the use of emotions in HRIs. These examples, coupled with the lack of guidelines on requirements, boundaries, and the appropriate use of emotions in HRI, give rise to a vast number of ELS implications that the second section addresses. Recent regulatory initiatives in the European Union (EU) aim at mitigating the risks posed by robot technologies. However, these may not entirely suffice to frame adequately the questions the use of emotions entails in these contexts.

Søraa, R. A., Fosch-Villaronga, E., Quintas, J., Dias, J., Tøndel, G., Sørgaard, J., … & Serrano, J. A. (2020). Mitigating isolation and loneliness with technology through emotional care by social robots in remote. In Ray, K. P., Nakashima, N., Ahmed, A., Ro, S. C., Shoshino, Y. (2020) Mobile Technologies for Delivering Healthcare in Remote, Rural Or Developing Regions, 255

This book chapter explores how experiences of isolation and loneliness in remote areas can be mitigated through social robots' emotional care. We first discuss the concept of being social and how that notion is changing with rapid digitalization. The research for this chapter zooms in the context of remote regions, characterized by vast geographical distances between cities, public services, and people's homes, in concrete, in northern and southern European regions. Then, the chapter discusses the Scandinavian term 'welfare technology,' and investigate different technological advances aiming towards bridging the gap loneliness poses. We propose the use and development of social robots equipped with emotional care support as a way of mitigating loneliness, given that the users do not experience the implementation of the technology into their daily life as paternalistic. We close the article with reflections on the consequences of such a sensible choice.

Sætra, H. S. & Fosch-Villaronga, E. (2021) Healthcare Digitalisation and the Changing Nature of Work and Society. Healthcare, 9(8) 1007, 1-15.

Digital technologies have profound effects on all areas of modern life, including the workplace. Certain forms of digitalisation entail simply exchanging digital files for paper, while more complex instances involve machines performing a wide variety of tasks on behalf of humans. While some are wary of the displacement of humans that occurs when, for example, robots perform tasks previously performed by humans, others argue that robots only perform the tasks that robots should have carried out in the very first place and never by humans. Understanding the impacts of digitalisation in the workplace requires an understanding of the effects of digital technology on the tasks we perform, and these effects are often not foreseeable. In this article, the changing nature of work in the health care sector is used as a case to analyse such change and its implications on three levels: the societal (macro), organisational (meso), and individual level (micro). Analysing these transformations by using a layered approach is helpful for understanding the actual magnitude of the changes that are occurring and creates the foundation for an informed regulatory and societal response. We argue that, while artificial intelligence, big data, and robotics are revolutionary technologies, most of the changes we see involve technological substitution and not infrastructural change. Even though this undermines the assumption that these new technologies constitute a fourth industrial revolution, their effects on the micro and meso level still require both political awareness and proportional regulatory responses.