Interested visitors were given a first taste of interacting with a virtual counterpart at the 13th Venice Architecture Biennale in 2012. The installation there was designed by Wolfgang Tschapeller upon an invitation from the Commissioner, Arno Ritter. Tschapeller and his team mounted a mirror down one side of the long Austrian pavilion. Animated 3D figures, digitally created on the basis of real people, were projected onto the opposite side. The mirror imagery provided a place of encounter for real and virtual guests. Through its PEEK Programme for arts-based research, the Austrian Science Fund FWF funded the interdisciplinary design of this multifaceted experiential space, named INTRA-SPACE. It offered a kind of test zone for speculating about the relationships between virtual figures and real people and for providing first-hand experience of such relationships.
An experience of space in the moment
The principal investigator, Wolfgang Tschapeller from the Institute for Art and Architecture at the Academy of Fine Arts Vienna, set up an interdisciplinary team of 18 people covering the fields of motion capture, 3D visualisation, computer graphics and animation, architecture, architectural theory, artificial intelligence, neuro-science and dance. At the core of INTRA-SPACE lies the notion of intra-action, coined by the theoretical physicist and feminist philosopher Karen Barad. Differing from interaction, intra-action considers processes, identities and objects as not being pre-defined but as something that emerges on the spur of the moment. This entwining is what Barad calls intra-action: “We were inspired by Karen Barad’s concept and tried to translate it into a spatial system of reference in which virtual bodies and real bodies can mingle”, says Christina Jauernik, an architect and arts-research collaborator in the project, describing the underlying concept. Based on an exploration of the nature of real-time measurements, the team members spent two-and-a-half years working together, on site or remotely, on this space: an experimental model for intra-action.
Encounters without technical assistance
A central element in the project was the co-operation with the Institute for Visual Computing and Human-Centered Technology at TU Wien (Michael Wimmer, Christian Freude) and with Nils Hasler and his research team at the Max-Planck Institute in Saarbrücken. The scientists there developed an experimental motion-capture system that does not require body-suits with motion-tracking markers as currently utilised for computer animation. The researchers used industrial cameras and algorithms which compute motion data from the difference between static space and moving people. They created nine different “diaphanous” (i.e. transparent, transient) figures, controlled in real-time via the motion and behaviour of each real-life counterpart: an elderly lady, the young girl “Carla”, “Bob”, a man in his mid-thirties, an old man, etc. These virtual figures have a few superhuman tricks up their sleeves: they can melt into each other, stand upside-down on the ceiling, transform body parts into objects, zoom, morph, disappear and pop up somewhere else.
Individual and collective practice sessions
In two consecutive series of experiments, the team developed scenarios for the encounter between one or several real and virtual individuals. Since the virtual figures are fully defined only through their encounters with real-life humans, every intra-action in the INTRA-SPACE is different, in the sense of Barad’s definition. First encounters juxtaposed one guest with one virtual counterpart – acting as a doppelgänger. In the second series, up to five live guests met virtual figures that interacted with the guests but also with each other. Thanks to an artificial intelligence component, the virtual figures came into their own and did not just mirror the guests’ movements. The sensory experience was modified further by moving the human eyes to the knee, for instance, with the help of the computer, which altered the perspective on the virtual counterpart.
“We deliberately opted for staying close to the human body and using this immediacy to build a relationship – we also tried to keep the spatial reference system as easily accessible and casual as possible”, explains Christina Jauernik in the interview with scilog. INTRA-SPACE was conceived as a prototype. It could be installed in a public area or in a private home, in a gallery or, as is currently the case at the Institute for Art and Architecture, as a learning tool for students. School classes and groups of refugees have also visited the space and experienced its impact. The project has given rise to a functioning infrastructure allowing encounters to play out on a physical and emotional level. “The close daily collaboration with “Carla”, for instance, transformed my wardrobe”, notes Jauernik. “I noticed that we worked together better in certain outfits for reasons of contrast and colour scheme. Such anecdotal insights provide first indications of how the real and the virtual persons become intertwined in our project.” In the last project phase, individual sessions provided an opportunity for feedback to identify potential fields of application. The system might be used in perception psychology, for rehab in physiotherapy sessions, in behavioural research, in interaction design or in the neuro-sciences.
Personal details
Christina Jauernik studied art and architecture at the Academy of Fine Arts Vienna, at the Berlin University of the Arts as well as contemporary dance at the Hogeschool voor de Kunsten Amsterdam and choreography as a site-specific art at the Dartington College of Arts, UK. She was a creative co-worker and doctoral student in the research project INTRA-SPACE headed by Wolfgang Tschapeller.
Project website: https://intraspace.akbild.ac.at/
Comments (0)
Currently no comments for this article.