Beyond the mainstream
February 2015. Concertgebouw Amsterdam. On the programme: the Alpine Symphony of Richard Strauss. A 45-minute piece, one hundred musicians of the renowned Royal Concertgebouw Orchestra on stage. On this evening, the audience witnesses a world premiere: this is the first time a computer is used that can recognise and understand music. The machine discerns what the orchestra just plays and offers additional information, as for instance the musical techniques Strauss used to create the respective sound colours. Equipped with tablets, the audience is able to follow the score and the additional information in real time.
A machine that learns
The synchronisation programme required for that purpose was developed by a research team around Gerhard Widmer – at the scientist’s two laboratories at the Department of Computational Perception at the University of Linz and at the Austrian Research Institute for Artificial Intelligence in Vienna. “The computer processes audio signals. Within fractions of a second it has to realise that music is being performed and which piece it is. The score is stored in the computer’s memory, but it also has to be able to understand the performance in real time”, explains the information scientist. This involves a huge computing effort: 45 minutes of music correspond to 240 million numbers or 3.8 billion bits. And the greatest challenge: not only must the software recognise music, it has to be able to follow the music in real time. The hitch: this is not something that can be programmed. “We feed the machine raw data, and it has to do the learning itself. This is what machine learning and deep learning are about, being able to feed the computer more and more raw data that it learns from”, is how Gerhard Widmer describes the direction where his discipline of Artificial Intelligence is heading. <iframe src="https://player.vimeo.com/video/136184873" width="650" height="365" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
(Video in German, Copyright: FWF/Ethan Vincent)
New audiences for classical music “With our programme, concert halls and operas can enrich their live streams with additional information”, notes the Austrian information scientist about possible applications. The underlying idea: classical music wants to attract new and younger audiences. This is also the declared aim of the “PHENICX” EU project under whose auspices this system try-out in Amsterdam took place in February.
START signal
Widmer’s research in that direction began with the START programme, a grant which the Austrian Science Fund FWF allocated to him in 1998. “This award made my research an accepted and relevant topic”, recalls the scientist who holds a chair at the University of Linz and adds: “It was a close decision at the time, because the jury had doubts about the realisation of my project. By now, I guess, no one is doubting anymore,” he smiles,
alluding to subsequent prizes in recognition of his achievements: in 2009 Widmer received the Wittgenstein Award – Austria’s most highly endowed science award – which enabled him to continue refining his methods. Yet another great distinction was to follow. Early in 2015 Widmer was awarded one of the internationally renowned ERC Advanced Grants by the European Research Council for his Con Espressione project.
Con Espressione
In this project, Widmer wants to generate a synthesis between his two fields of research: he wants computers to be able to perceive music on the level of expression. “They should be able to recognise emotions, understand whether the music is cheerful, light, aggressive or hesitant. I would like to build a prototype which interacts with musicians, which relates to the interpretation, the tempi, the expression and intention and plays along”, is how Widmer describes his ambitions. This being said, the scientist is certain that human beings will never be replaceable: “Human perception is too complex.”
“The boy needs to go to grammar school”
Having grown up in a village of 600 inhabitants near Bregenz, the 54-year-old researcher was not predestined for a scientific career. He was even the first boy from the village to attend grammar school. “An out-of-town couple who were teachers said: ‘The boy needs to go to grammar school’”, recalls Widmer. Nor was it the done thing in the village to start playing the piano at 7, as he did. Such was his talent that he soon looked to be on his way to a career as a professional musician. At 15, however, he realised that he was too lazy to practice. And half an hour of practice a day will not make anyone a “world champion”.
Computer and logic
Choosing a field of study was not obvious for this young man of many interests. At school, Widmer was good at all subjects. While browsing through a study guide on computer science of the Vienna University of Technology (TU Wien), he was particularly attracted by two terms: computer and logic. At the time, a computer was a thing of mystery to him. “In 1979, no one had ever seen a genuine computer”, grins the researcher who now resides in Vienna. He describes the scientific career that ensued as “accidental”. Obviously, anything he does comes easily to him. After his studies a Fulbright grant took him to the US where he found more time again to play the piano and – as he recounts with some pride – played in the Black Music Ensemble of jazz musician Richard Davis. He returned to Vienna to the institute from which he graduated and, in 1995, obtained his venia legendi at TU Wien. When did he decide to become a scientist? It just turned out that way: “I never consciously took the decision for science and then, once a scientist, I never decided to do anything else”, observes Widmer.
Quality over quantity
Widmer considers he owes his success to his versatility: “A broad education, broad interests and the ability to create links between different things.” These are
qualities he also appreciates in his team members. “One should engage with topics one considers important, even if it is not part of the mainstream initially”, is what he believes to be an important quality of a good scientist. Another thing he always recommends to his team: “One should focus on quality, not quantity. Your publications become your business card”, says Widmer and elaborates: “You have an international reputation. Your colleagues know exactly who publishes something of quality. In order to get funding such as an ERC grant you have to have a really good reputation. The sheer number of papers is not sufficient.“
All freedom to the team
Here are the qualities Widmer considers particularly important in his co-workers: “They should be creative and not afraid of anything; see it through even if something seems too difficult.” In order to foster that, Widmer gives them total leeway to follow up on their ideas. Accordingly, he perceives his role to be more that of a mentor than a leader. When recruiting, he attributes particular importance to making sure the applicants are socially compatible with the group. “My teams are like families. When you come to Linz to our institute, you see all doors wide open and people standing together discussing things”, so Widmer describes the atmosphere he considers of such importance. “If everyone were to sit in their little cubicles and take no interest in the others that would be dreadful!” At the moment he is looking for team members for his ERC project. Not a simple task, since the applicants have to demonstrate remarkable interdisciplinary skills. “People who have a serious background in machine learning and deep learning, in statistics and information theory – and in classical music”, is how he describes the requirements. Not easy to find. But they exist. He himself is ample proof of that.
Gerhard Widmer is a full professor and head of the Department of Computational Perception at the Johannes Kepler University Linz. He is the founder and head of the Intelligent Music Processing and Machine Learning group at the Austrian Research Institute for Artificial Intelligence in Vienna, OFAI. A successful pianist until his teenage years, he then decided to study information science at the Vienna University of Technology. During a Fulbright grant in the USA he lived his passion for music by playing in the Black Music Ensemble of jazz master musician Richard Davis. Apart from numerous renowned scientific distinctions he was awarded the 1998 START award, the Austrian Science Fund FWF’s grant for young researchers, the FWF’s Wittgenstein Award (dubbed Austrian Nobel Prize) in 2009, as well as an ERC Advanced Grant by the European Research Council early in 2015.
More information
