# What does "Convolutional" signify in "Convolutional Neural Network"?

The "Types of Neural Networks" wiki page doesn't even have the word "convolution" in it, and yet there's an entire Javascript library based on Convolutional Neural Networks (CNN).

So, what makes CNNs "convolutional"? When I hear "convolution", I think of either smearing/blurring together (as in the "convolve" tool in programs like Photoshop), or complexity/tangling (e.g., "a long and unnecessarily convoluted explanation").

When I think of neural networks, I think of networks with distinct layers, where neurons from one layer only connect to neurons of the next layer. To me, "convolutional" suggests that the neurons are more intricately connected somehow—or "tangled up". But I see nothing in the javascript library's documentation to suggest that this is the case.

The background section in the "On Complex Valued Convolutional Neural Networks" master thesis suggests that CNNs are what has made neural networks actually useful for tasks like computer vision and face recognition. Why remains unclear, however, the term "convolution operation" was mentioned.

Guberman, N. (2016). On Complex Valued Convolutional Neural Networks. arXiv preprint arXiv:1602.09046.

• It is the mathematical meaning which is related to the Photoshop convolve tool. Mar 14, 2016 at 20:03
• i.e. each layer of the network performs a particular convolution on its inputs.
– honi
Mar 15, 2016 at 16:40

Convolution is a technical term, and comes from the signal processing field. Unfortunately, it is easily confused with "convoluting". Effectively, "to convolve" means to pick a small "window", slide it across your signal, and compute the same function for each step.

In CNNs, when applied to classifying, for example images, it means taking a small window, of say 3x3 pixels, sliding it across the whole image, and feeding the small sections of the larger image under the window into a neural network. The outputs of the network are then combined into a new layer and can then be fed into another NN layer for classification.

In the example above, the window is 3x3 pixels wide, and the step size is 1. Depending on the application, it's possible to have larger/smaller windows and step sizes.

The resulting layer can be said to be the result of convolving the original image with a neural network.

When using convolution filters in, say Photoshop, it's using a similar method where it takes sliding windows of pixels and computes a function (for example to achieve blur, the average RGB values of the window pixels) and creates a new image out of those. You can then say that convolving an image with a pixel average function results in a blurred image.

• And there are different functions that can be swapped in for the pixel-average function, resulting in things like edge detection. (I've heard this function called "the filter kernel".
– DJG
May 19, 2016 at 21:29
• Minx, yes exactly May 19, 2016 at 21:54
• @Justas In the diagram, there is a circle below the circle with the $\sum$ in it. This circle has a stretched-out "S" symbol. What does it mean? Jul 19, 2022 at 3:36
• @user110391 the S symbol signifies a non-linear activation function through which the sum is passed. Depending on the use case, the S could be the sigmoid, tanh, or RELU function, among other possibilities. Jul 19, 2022 at 20:06
• @Justas Thanks :) Jul 19, 2022 at 21:02