Neural networks are commonly used today to analyze complex data – for instance to find clues to illnesses in genetic information. Ultimately, though, no one knows how these networks actually work exactly. That is why Fraunhofer researchers developed software that enables them to look into these black boxes and analyze how they function.
The software recognizes which parameters a neural network uses to make decisions.
It is of interest for any area in which data such as text, images and signals are automatically processed and evaluated by neural networks.
How neural networks function
Neural networks function, as their name suggests, based on the same principle as the brain. Just as various areas of nerve cells, the neurons, cooperate in our brain, mathematical units work together in the artificial neural network. A photo that, to a computer, is initially only an unordered cloud of pixels, is systematically analyzed by these groups of artificial neurons. Some neurons recognize edges while others recognize corners. Bit by bit, the image becomes clear to the computer. Ultimately, the outcome of this neural computing operation is a probability value – the computer calculates, for example, how likely it is that a face actually does correspond to that of the person being sought.
Machine learning enables customized cancer treatments
This is important, for instance, in detecting diseases. We already have the capability today to feed patients’ genetic data into computers – or neural networks – which then analyze the probability of a patient having a certain genetic disorder. “But it would be much more interesting to know precisely which characteristics the program bases its decisions on,” says Samek. It could be certain genetic defects the patient has – and these, in turn, could be a possible target for a cancer treatment that is tailored to individual patients.
The researchers’ method allows them to watch the work of the neural networks in reverse: they work through the program backwards, starting from the result. “We can see exactly where a certain group of neurons made a certain decision, and how strongly this decision impacted the result,” says Samek. The researchers have already impressively demonstrated – multiple times – that the method works.
“So you can see how important it is to understand exactly how such a network functions,” says Samek. This knowledge is also of particular interest to industry. “It is conceivable, for instance, that the operating data of a complex production plant could be analyzed to deduce which parameters impact product quality or cause it to fluctuate,” he says. The invention is also interesting for many other applications that involve the neural analysis of large or complex data volumes. “In another experiment, we were able to show which parameters a network uses to decide whether a face appears young or old.”
According to Samek, for a long time banks have even been using neural networks to analyze bank customers’ creditworthiness. To do this, large volumes of customer data are collected and evaluated by a neural network. “If we knew how the network reaches its decision, we could reduce the data volume right from the start by selecting the relevant parameters,” he says. This would certainly be in the customers’ interests, too.
Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI