Is there any relation between graphics/multimedia and machine learning (ML)?

The two very common buzzwords in modern computing are a) graphics and multimedia and b) artificial intelligence and machine learning. Apparently, these two domains appear to be perpendicular to each other in terms of what get achieved. Activities around graphics and multimedia peaked earlier, in the last decade of the 1900-s as well as the first few years of the current millennium. This resulted in specialized hardware dedicated specially for the domain. At the core of this were the so called Digital Signal Processors (DSP-s) which later also came to be known as Graphics Processing Units (GPU-s), specialized processors designed to accelerate graphics rendering and multimedia signal processing.

Artificial Intelligence started receiving renewed attention soon after that and with major players driving the field, words like 'Machine Learning', 'Neural Networks', 'Deep Learning' have become the buzzwords of today. Is there any relation between graphics/multimedia and machine learning (ML)? Why else do we hear people saying that GPU-s are essential for carrying out deep leaning computations at reasonable speed --- most popular ML installations will recommend having GPU-s to be present in your hardware. How do GPU-s help in making ML computations faster?

To answer the question, we must first understand what the basic computation requirements for ML are. If you look into a typical ML based solution, the maximum computational need is during the 'training phase', usually based on some version (shallow, deep, convolutional, etc.) of an Artificial Neural Network or Neural Network for short. In digital signal processing, almost all key operations (called transformations, like Fast Fourier Transform, FFT, Discrete Cosine Transform, DCT, and so on) are based on matrix multiplication. A matrix multiplication in turn is basically a series of 'dot-products', i.e., multiplication of pairs of numbers from two same-sized arrays and adding each product --- Σ(ai * bi). This means that dot-product, or 'multiply-and-add' is a fundamental necessity. Naturally, the DSP-s and GPU-s are therefore designed to carry out such an operation as fast as possible. In other words, DSP-s and GPU-s allow computation of a dot-product to be carried out in the least possible time. Normal CPU-s do are not designed for that and hence the DSP-s/GPU-s are considered special purpose processors. Quite often, internally a DSP/GPU achieves this by allowing for a good number of multipliers that can operate in parallel. The power (and hence price) of a DSP/GPU depends significantly upon how many such multiplications can be carried out in parallel.

Let us now look at a very simplified Neural Network consisting of only one input layer and one output layer as shown in the figure below. More complex networks have a few intermediate layers, many more nodes in the input layer and the output layer, and so on.

If you look at the computation carried out closely, you basically see series of 'multiply-and-add', one for each row. The number of rows is determined by the number of outputs which in most cases is significantly lower than the number of inputs. However, an individual 'multiply-and-add' needs as many multiplications as there are rows and we all know that that the number of rows, i.e., the number of inputs can be pretty large, going into tens of thousands if not millions. Clearly, to achieve the computational purpose, each 'multiply-and-add' must happen in the minimum amount of time. Recall that a DSP/GPU is built as that as a prime objective. Hence, GPU-s have become almost a necessity for ML computation, especially during the training phase.