A sleep mode for Siri and Alexa
It was a good ten years ago that they were awakened from their slumber – special computer methods known as neural networks, which, like the human brain, consist of interconnected neurons and are capable of learning how to solve complex tasks under their own steam. At the beginning of the millennium, neural networks were neglected by the scientific community. But actually, they are just a mathematical approach for reproducing functions. As early as the 1990s, researchers were able to show that neural networks can, in principle, learn almost any function, a feature that has many practical applications. Nevertheless, the artificial intelligence research community did not pick up on it in those days. “What was missing then was computing power, enough data and good software tools,” explains Franz Pernkopf of the Graz University of Technology. “That changed in 2010, and the field has been booming ever since. “ As a result, the performance of many AI applications has increased massively and has invaded our everyday life with voice-controlled services such as Siri or Alexa.
Reduce computing power
Neural networks still require a great deal of computing power, however, and the calculations are often outsourced from the user’s device to the cloud. The server farms run by cloud providers often use so-called GPUs for this purpose, which were actually developed for resource-hungry video games and are now heavily used in bitcoin mining. For some fields, such as self-driving cars or battery-powered devices, this massive computational requirement is a drawback. In a research project funded by the Austrian Science Fund FWF, Franz Pernkopf's group therefore sought alternative approaches to reduce the complexity of the calculations.
Not useful everywhere
“Neural networks do not make sense for every task,” Pernkopf says by way of introduction. If a physical model can describe the behaviour of a system well, then it is better to use that model. Neural networks kick in where the tasks are difficult to grasp. Pernkopf gives the example of recognising a cow in a pasture. “It is not so easy to define precisely what a cow looks like. In such a case, neural networks are very useful.” If a neural network is fed with a sufficient quantity of images of cows, it will eventually be able to recognise a cow in an image it has not seen before.
As a rule, neural networks use many more parameters than are actually needed. Pernkopf's team therefore looked for ways to reduce the complexity of artificial neural networks without compromising recognition rates. A neural network consists of a handful of components, and there are many ways to interconnect them. “We tried to develop automatic methods to find the most efficient network,” says Pernkopf.
Calculating with smaller numbers
Another starting point is the level of computer hardware. Today's PCs use 32 or 64 bits for addition and multiplication. With 32 bits, over four billion numbers can be represented. The problem is this: the computer treats every number as if it were of the order of four billion. Numbers of that order of magnitude are not required at all for many applications. “We found that we can reduce these bit widths without losing performance,” says Pernkopf when reporting on the current results. “If you compute with 8 bits instead of 32, you have immediately cut the computing operations to only a quarter.” He notes that the team even went so far as to calculate with only one bit instead of 8, with amazingly good performance in certain areas.
Pernkopf's team scored a big hit when they managed to represent the parameters as probability distributions instead of exact numbers. “We were the first to do that,” says Pernkopf, underlining the elegance of the new approach because it simplifies the search for the right parameters.
When should a system wake up?
It is an abstract result whose theoretical character flows from the new field of research. “When we submitted the funding application for the project, there was very little to be found in the literature,” notes Pernkopf. Soon afterwards, he says, publications on the subject began to appear one by one. Hence the project, which ran for four years and ended in 2020, was able to carry out truly pioneering work. It was conducted in cooperation with the University of Heidelberg, whose focus was more on computer hardware, while the researchers in Graz concentrated on machine learning aspects.
The basis for practice
Various follow-up projects, such as those funded by the FFG Research Promotion Agency and another international project with the German Research Foundation DFG, are now intended to bring the theoretical results into the sphere of application. One application case, however, was investigated during the basic research phase. It involved the recognition of keywords to awaken speech recognition systems from standby mode. “If I were to run speech recognition software permanently on a smartphone, the battery would run out after no more than an hour because it requires so much computation power,” Pernkopf explains. What is needed is a leaner, more resource-efficient system that has to recognise only a few words as stimuli – like a sleeper whose awareness is vastly curtailed. This would save a lot of energy.
Pernkopf is convinced that neural networks, and specifically resource-efficient systems in battery-powered devices, will continue to spread in our everyday lives. What he does not believe in, however, is surrendering dominance to artificial intelligence: “Human beings will not be replaced completely.”
Personal details
Franz Pernkopf is an electrical engineer and he conducts research at the Institute for Signal Processing and Speech Communication at the Graz University of Technology. He is a multi-award-winning researcher, who is particularly interested in machine learning and pattern recognition, especially in the fields of medical technology and speech signal processing. The international research project “Resource-efficient deep models for embedded systems” (2016-2020) was funded by the Austrian Science Fund FWF with EUR 214,000.
Publications
Rock J., Roth W., Toth M., Meissner P., Pernkopf F.: Resource-efficient Deep Neural Networks for Automotive Radar Interference Mitigation, in: IEEE Journal of Selected Topics in Signal Processing, Vol. 15, 2021
Roth W., Schindler G., Zöhrer M., Pfeifenberger L., Tschiatschek S., Peharz R., Fröning H., Pernkopf, F., Ghahramani Z.: Resource-Efficient Neural Networks for Embedded Systems, in: Journal of Machine Learning Research, revised 2021
Peter D., Roth W., Pernkopf F.: Resource-efficient DNNs for Keyword Spotting using Neural Architecture Search and Quantization, in: 2020 25th International Conference on Pattern Recognition (ICPR) 2021
