Exploring communication in music ensembles
The whole is often more than the sum of its parts, a tenet which is particularly true of classical music. In order to be forged into an organic whole, ensembles and orchestras depend on the complex communication skills of individual musicians. The element of unpredictability in a performance, caused by creativity and the striving for uniqueness, seems to contravene the requirement of harmonic, flawless interaction between the ensemble members. Researchers still need to understand precisely how this contradiction is resolved in musical practice. The psychologist and trained pianist Laura Bishop from the Austrian Research Institute for Artificial Intelligence (OFAI) is now exploring this issue in a project financed by the Austrian Science Fund FWF. She has developed empirical methods to analyse the routes of communication in classical music performance.
Recording the movements in duo performances
“We started investigating musicians playing together in a duo”, says Bishop. “For the analysis we used motion capture and eye-tracking.” On the premises of her research partner, the Vienna University of Music and Performing Arts, a room was converted into a laboratory and equipped with infra-red cameras. The musicians wore vests studded with reflective buttons and special glasses to record their eye movements. “We then gave them a piece of music written specifically for that purpose and asked them to practise it”, Bishop relates. “The piece wasn't particularly difficult to play, but it featured a section without a defined rhythm, where the timing becomes irregular.” The test was conducted with 21 pairs of musicians. The researchers took particular interest in how the movements in the relevant section of the piece changed. One of the outcomes was that the movements of the musicians became more and more aligned the longer the piece was practised.
A computer that imitates people
Based on this experimental setup, Bishop and her team now intend to investigate how the performers’ behaviour changes when they play with a computer as their duo partner. In cooperation with other departments in her research institute, they are developing a computer system that is capable of listening and responding to tempo changes and errors. This is still a challenging task, explains Bishop: “Our current system is able to play louder or softer, change the articulation and take on board some additional information found on the sheet music, such as a crescendo - a gradual increase in volume. The system therefore behaves like someone who has a certain knowledge of the conventions of classical music.” One of the challenges is variability: human performers make small mistakes while playing, but they are typically human mistakes. “Mimicking this is difficult. We are still working on it”, explains the researcher. Another challenge is that usually one player leads and one follows, when a piece is performed. “The person who plays the melody usually takes the lead, the other person follows.” The computer system in its current form is able to follow, but not take the lead. All of the described elements only constitute preparatory work, however. “Our goal is to find out what it takes for a person to feel that they are playing with a human being and not with a computer”, notes the researcher. Bishop confirms that this has strong similarities with the Turing Test, a famous thought experiment from artificial intelligence research. The Turing Test involves a human being talking to a computer. If the test person cannot tell whether they are talking to a human or a computer, the computer has passed the test.
A better understanding of people as ultimate goal
Laura Bishop's research is not really about replacing people with computers to perform classical music. The computer methods are primarily designed to analyse human behaviour more precisely, and human behaviour is what the researcher would like to understand better. “People aren't perfect, they're different from one another. This creates some uncertainty about what exactly is going to happen. A piece of classical music can sound different in every performance.” This is part of what makes it exciting, Bishop says. While not a perfect replacement, a computer system that can play classical music together with a human being is nevertheless interesting for practical reasons. “The people who would benefit most from such a system of accompaniment are music students who want to practise a piece. Practice time with partners is always limited, a computer system would be a great help in this respect.” Bishop, who hails from Canada and studied piano at the Conservatoire there, emphasises the great importance of Vienna as a location for her research. She collaborates closely with the Institute for Musical Acoustics at the University of Music and Performing Arts in Vienna.“There is hardly any comparable institution in the world that has access to so many people who make music professionally. This is a great opportunity for us.”
Personal details Laura Bishop is a post-doctoral researcher at the Austrian Research Institute for Artificial Intelligence (OFAI) in Vienna. She is interested in communication in musical ensembles, coordination between people and scientific approaches to creativity.
Publications
