Demystifying the "Hidden Layers" in Convolutional Neural Networks

I spend a lot more time than you'd think talking informally about machine learning and Deep Learning with friends, family and co-workers. One thing that comes up a LOT is the idea that 'things are hidden'. Generally this means that the person I'm talking with doesn't actually understand the terms they are hearing. The big one is the idea of a 'hidden layer'. This makes it sound as if there is a layer that is invisible; which is not the true state of the network. In the post below I won't go into network topology, but talk about some of the terms.

Convolutional neural networks (CNNs) have revolutionized computer vision tasks, enabling remarkable achievements in image classification, object detection, and more. As you delve into designing a CNN, you might come across the term "hidden layers" and wonder why they are called "hidden" when you have full control over their shape and function. In this blog post, we'll unravel the mystery behind these "hidden layers" and explore their role in the CNN architecture.

In a CNN, the term "hidden layers" refers to the intermediate layers situated between the input and output layers. These layers are labeled as "hidden" because they are not directly observable from the input or output of the network. However, as the designer, you have the power to shape and define their characteristics according to your requirements.

Let's explore the different layers typically found in a CNN:

First the Input Layer:

• The input layer receives the raw data, such as an image, and serves as the starting point for processing.

Then the so-called Hidden Layers:

• Convolutional Layers: These layers perform feature extraction by applying convolutional operations to the input data. They capture local patterns and hierarchies of features in the data.
• Activation Functions: Activation functions introduce non-linearity, enabling the network to learn complex relationships between features.
• Pooling Layers: These layers downsample the spatial dimensions of the feature maps, reducing complexity and providing translation invariance.

... and finally the Output Layer:

• The output layer represents the final layer of the network, responsible for producing the desired output, such as classification probabilities or regression values.

A note about designing the Hidden Layers:

As the designer, you have complete control over the shape and function of the hidden layers. While the term "hidden" might initially seem perplexing, it does not imply that you are unaware of their characteristics and to a degree machinations. In fact, you possess the ability to define the layer shapes, the activation functions employed, and the connectivity between layers.

• The term "hidden layers" in a CNN refers to the intermediate layers between the input and output layers. Although the layers are not directly observable from the input or output, you have the authority to shape their structure and functionality. By defining the convolutional layers, activation functions, pooling layers, and other specialized layers, you guide the network's ability to extract meaningful features and make accurate predictions.

This is super simple. I could spend a whole post talking about each layer of a neural network (maybe I will). This understanding of the terminology and the purpose of each layer in a CNN will hopefully help people who are not in the immediate world of designing these tools feel left out of the conversation.