The consensus decision making process of a data science machine learning AI model is often referred to as the AI Black Box by researchers and users. In this article, we will explain it in the most simple and at the same time accurate way. We live in a world that is already augmented. Imagine going out with your friends and somebody asks you a question to which you do not know the answer. If you have a smartphone you can find it within seconds. But this type of augmentation is just a primitive one. We can also think of more advanced applications like Siri, face recognition systems, or autonomous vehicles. Compared to primitive ones, such systems do not require our explicit human direction. When it comes to advanced Artificial Intelligence (AI) models, which get trained over time, this absence of human direction raises a number of concerns.
Let’s try to address them by observing the advanced AI Neural Networks (NN) that are extremely powerful. In such deep learning approaches, models learn by observing hundreds, sometimes even thousands different datasets. If you want a neural network model to recognize a picture of a car, you show it thousands of pictures of cars with different characteristics, and slowly, you train the AI model to become extremely powerful in the specific task. The problem is that we do not know or do not understand how the model has internally oriented itself to start recognizing what a car is. Therefore, we don’t have a straight insight into how it reaches its conclusions and findings.
Let’s take another example to explain it further: An AI model which recognizes pictures of humans.
In such a rule-based system you need to “feed” data to the model of human images. In this way, it gets trained to understand human body characteristics and body figures. The question here goes down to “Why did analysts decide to place this component on the machine using the characteristics of a human?”. The AI model is trying to place components on a printed circuit board. However, there is a traveling salesman problem; it is the optimal path for determining the way that specific human characteristics interact with the printed circuit board. A genetic algorithm can be used to evolve that problem. It cannot be asked why it made these choices. But we can ask the front end of the system why the rules generated the end results…
…But the AI black box problems will still be there. There is some kind of algorithm that is used and has certain inputs and outputs. You can look at both sides of those, but you have no idea what happened in the middle. Sometimes it can be very uncomfortable to trust such a model. Despite whichever ability of the AI model to recognize objects or forecast the weather or predict the stock market, such models are a black box to their designers. You might get the right answers, but for the wrong reasons. The goal is to open this AI black box and understand what artificial intelligence is doing and why.
Want to learn more? Check out the video of ZDNET
