Human-centred artificial intelligence

“There used to be this joke,” says Andreas Holzinger. “You go to see your bank advisor for a loan. The advisor looks at his computer screen and says he can’t approve your loan. When you ask why, he replies: because the computer says so.” This is how Holzinger circumscribes a problem that has been around for a long time, but has recently taken a back seat owing to the great successes of artificial intelligence (AI) through neural networks. Algorithms in cars already recognise traffic signs, mobile phones understand our language, and translation tools deliver amazingly accurate results – and all of this happens completely autonomously, without human involvement. In certain areas of application, however – medicine being one of them – this “fully automated” black-box approach is raising more and more legal and ethical issues. In a project funded by the Austrian Science Fund FWF, Andreas Holzinger is exploring how humans can once again become part of the decision-making process.

Including human experience and understanding

Holzinger says that the problems are home-made: “Almost exactly ten years ago, I gave a lecture at Carnegie Mellon University in the US, a top university in the field of AI, where I put forward the idea of adding human intervention to AI applications.” Holzinger was disappointed, however, by how little resonance this received from the research community: “At the time, they wanted to make everything fully automatic – without any human involvement.” Over time, neural networks became more and more powerful but also more and more complex, and thus more “opaque”. The reason is that they are not coded by humans, but learn autonomously from huge amounts of data. This approach has the side effect that in most cases it is no longer easy to understand exactly how the algorithms arrive at their decisions.

The EU’s new data protection directives are now causing AI specialists to rethink the approach. “The General Data Protection Regulation contains something like a right to explainability. So if an autonomous AI does not approve your loan, you have the right to ask about the underlying criteria on which this decision is based,” says Holzinger. For AI developers this is a problem, however, because the research community has so far mainly worked on algorithms for automatic and autonomous decisions and was not really prepared for this.

The concept of “Explainable Artificial Intelligence” promises a solution, as it is designed to make AI decisions more transparent. Intensive research is currently being devoted to this area. One variant of explainable AI is implementing the possibility to ask “if-then” questions. This requires making so-called counterfactual statements, i.e. fictitious “what-if” assumptions designed to question decision-making processes. Holzinger notes that such interventions require interactive user interfaces, and for these you need to have an idea of how good the explanations of decision-making processes are.