When algorithms result in exclusion
Technological progress makes many things easier and shows us a way forward. But what if automation and artificial intelligence fail to take the real world into account? “There is a great deal of talk about the potential of technology without looking at the status quo,” notes Katta Spiel from the Human-Computer Interaction Group at the Department of Computer Science at the Vienna University of Technology (TU Wien). The result is a failure to reflect social diversity, while norms and stereotypes instead become entrenched. This can become a problem wherever technology and human beings interact closely, i.e. instances where technology connects directly with the body, as is the case with wearables, smart tattoos or intelligent clothing.
At this human-machine interface, Katta Spiel is conducting research into methodology, which means the researcher is challenging entrenched assumptions in technology and in design processes. There is one fundamental and, perhaps, provocative question, as Spiel herself says, underlying her work and the Firnberg project on “marginalised bodies” funded by the Austrian Science Fund FWF: what does it mean to be considered a human being or not? Spiel is interested in what lies outside the norms of developers, and focuses on gender and disability. Her objective is to improve how human beings are represented in all their diversity, thereby preventing social exclusion. According to Spiel, marginalisation happens faster than one may think. One example relates to fitness trackers: people are encouraged to exercise more, but fitness trackers are mainly designed for those who are already fit. Spiel highlights the fact that the design of the wristbands alone excludes obese people. But it is even a great deal more problematic – to stay with this example – that many people who use fitness trackers are afraid of not being slim enough to fit the norm. This promotes eating disorders.
The bias effect
Katta Spiel found another example in a literature study that was recently published. Researchers developed a t-shirt equipped with magnets to improve posture. The design was based on drawing black dots on white skin. “It's difficult for minority groups to be confronted with technologies that are developed by a particular demographic,” Spiel says. The standards of technology still address a healthy, rather affluent, white, western male and quite rarely reflect social diversity. This tallies with the fact that only 22 percent of professionals in the fields of artificial intelligence (AI) and data science are women. A Standford analysis of 133 biased AI systems across industries from 1988 to the present day found that almost one in two systems has a gender bias and one in four systems exhibits both gender and racial bias.
So how can design incorporate different needs and be mass-compatible without repeating exclusion? It begins with the collection of data. Spiel conducted a test by searching a variety of databases for gender coverage: “Although several gender identities are now recognised in Austria, there are still a huge number of databases that are unable to reflect this – until recently this included the central civil register.” Since more and more procedures have now become automated, this situation will eventually distort the available data. When women or non-binary individuals are underrepresented, this engenders problems in many areas of life, particularly in medicine, where they consequently lack reference values on which therapies should be based. The standard for many medical tests is still the male body.
Digital inclusion and DIY
Katta Spiel turns the tables when designing wearables herself. Instead of choosing as starting point a problem and addressing it with a lot of complex technology, Spiel takes guidance from quality and benefits. The scholar asks questions arising from different needs: what makes sense, when do I need something, when can I do without? And how does it feel to wear this object? A central aspect is that Spiel involves the users in this process. Such a participatory process gives rise to products that are technically very simple but effective. To give an example: Spiel is currently developing a set of “stimmies” using wool and magnetic threads and self-guidance. These stimmies can support people suffering from ADHD or autism, for example. They help regulate repetitive movements, such as foot tapping. They interact with the movements by creating patterns using light or vibration. “A magnetic strip surrounded by crochet work worn as a bracelet across which you move your fingers has been the most successful so far,” reports Spiel, who has already received several awards for her participatory work with autistic children.
When technological development moves too far away from social reality, it misses many opportunities – by excluding marginalised groups, entrenching norms, reproducing inequalities and postponing problem solving into the future. “We have to think about what we need right now and deal with the social effects. But we've known that for decades anyway,” Spiel says wistfully.
Project website: https://exceptional-norms.at/
Personal details
Katta Spiel is conducting research on human-machine interaction with a focus on gender and disability. The scholar studied media culture and computer science at Bauhaus University Weimar. In 2014, Spiel switched to the Vienna University of Technology (TU) and completed a postdoctoral year at the University of Leuven (Belgium). Spiel is currently conducting research on the topic of “Marginalised Bodies in Interaction Design” (2020–2023) in the context of a Hertha Firnberg Fellowship from the Austrian Science Fund FWF in the HCI Group at TU Wien. Katta Spiel has received several awards for participatory work with autistic children.
Publications
Katta Spiel: The Bodies of TEI – Investigating Norms and Assumptions in the Design of Embodied Interaction, in: TEI ’21: Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction 2021
Katta Spiel: Why are they all obsessed with Gender? — (Non)binary Navigations through Technological Infrastructures, in: DIS ’21: Designing Interactive Systems Conference 2021
