Deep Diving Into Deep Learning

Artificial intelligence (AI) algorithms beat a human chess master in 1997. It took almost another two decades for AI to beat a human Go champion in 2016. Deep learning, a branch of AI, made it possible.

The earlier AI algorithms were rule-based. They were designed to teach computers “what to do.” The deep learning algorithms that defeated the Go player is designed to teach computers “how to learn.” These algorithms can use a large set of data to learn on their own. It's changing the way AI operates.

Human Neural Networks

Deep learning is based on the concepts of the human neural network. In a neural network, neurons have the following components:

• Dendrites - It takes inputs from other neurons in the network. The inputs are passed as electrical impulses. There are multiple dendrites on a neuron.
• Soma - Soma is the cell body of the neuron. It is the control center of the neuron.
• Axon - It’s the output port of a neuron. The outputs are in electrical impulses.
• Synapse - It is a connection between two neurons.

Dendrites bring the flow of information to the neuron and the information flows away through the axon. The strengths of synapses determine the strengths of the electric signals between the neuron connections. Billions of neurons interconnecting with each other create a neural network of the human brain that can recognize patterns and make decisions.

Replication of the Neural Network Mechanism

Deep learning takes the ideas of the neural nets and represents them with nodes. The nodes can have multiple layers. Nowadays six or seven layers are common. But Microsoft has expanded that to 152 layers. A layer’s outputs are fed into the next layer as inputs.

The strengths of connections between the nodes are an important factor. Suppose, there are 4 nodes A, B, C, and D that connects to the next layer with 2 nodes X and Y. If X is strongly connected to A and C nodes and Y is strongly connected to C and D nodes, then X will have AC and Y will have CD properties. Now if the X and Y layer is connected to another set of nodes with different strengths of connections, they can create the next level of properties.

In a deep learning algorithm, once the neural network is set up, the initial connection strengths are randomly assigned. Then the input data is fed into the system to train the neural network.

During the training period, the algorithm looks at the result of the neural network, compares it with the data provided and calibrates the connection strengths to bring the error down. More data results in better calibrated deep learning algorithms.

Even after training, the calibration process continues during live data processing. So, if the conditions change, the neural network changes with it.

Computational Power for Deep Learning

Deep learning depends on matrix multiplication. As a result, instead of sequential processing, it’s possible to take advantage of parallel processing. So GPUs (Graphics Processing Units) are great for training neural networks. With hundreds of simple cores, GPUs can process information faster.

Initially, CPUs (Central Processing Units) were used. But the cost was enormous. Google Datacenter’s CPU based neural network cost $5 billion for 1K CPU server and 16K cores. Stanford Lab was able to set up an almost equivalent system for $33K using 3 GPU-accelerated servers and 18K cores. So GPU-base processing is currently the norm for deep learning machines.

Conclusion

Deep learning is having an enormous effect on our society. Self-driving cars, natural language processing, speech recognition, image recognition and more are benefiting from the advances of this technology. Using the ideas of human neural nets and harnessing the power of super-fast GPUs, deep learning is changing the way computers interact and help us.