Convolutional neural networks (CNNs) are most commonly associated with images, but they can also be applied to sequences or spectra. We will discuss the image application first. In chemistry and biochemistry image data include microscopy (both light and electron microscopy) and imaging mass spectrometry data. CNNs are made up of convolutional layers, these layers apply a convolution to segments of the image. During convolution the data from the image (pixel intensity) are multiplied by the corresponding position in the convolutional filter, or kernel. These values are parameters that are fit during training of the CNN. These products are then summed to determine the output of the convolution. The kernel slide across the image to determine the outputs at each location, the same kernel is used at every point in the image. The hyperparameters for kernels include the number of output channels (this will be related to the number of kernels used in the network), the kernel size or dimensions, the stride or how large the step of the kernel is as it moves across the image, and padding which determines how many extra pixels are added at the edge of the image. Early convolution layers tend to learn basic image features such as recognizing a vertical line while later layers learn more complex shapes related to the task. The initial input to the network can have different colors (for example R, G, B values in a color image). For microscopy this would correspond to the number of channels being imaged and in imaging mass spectrometry to the number of m/z features chosen for analysis or some other dimensionally-reduced representation of the m/z features.

Additional layers can be added between the convolutional layers. The most commonly used are pooling layers, these also slide a kernel across the image but instead of applying a convolution they take the numbers inside that window and either average or find the maximum value, depending on the type of pooling. Pooling layers provide a way to shrink the image, which can help reduce the number of parameters while still retaining the most important information in the image. Max pooling is more commonly used than average pooling. Pooling can either be done after every convolution or after a few convolutions have been applied in a row. Batch normalization layers (BatchNorm) can also be used after convolution layers to put kernel outputs on a similar scale to make training more stable. Generally we want the size of the image to decrease and the number of kernels to increase as we move deeper in the network.

After going through the CNN, the output from the last layer is flattened. Meaning it is turned into a 1D vector from each of the kernel outputs from the previous layer (which will be the dimensions of the image after convolution x the output size). This linear representation can then be passed to a one or more fully connected layers to be further processed before being passed to an output layer.

The following websites have some nice visualizations of how CNNs work if you want more information: https://medium.com/thedeephub/convolutional-neural-networks-a-comprehensive-guide-5cc0b5eae175 and https://poloclub.github.io/cnn-explainer/

1D convolutional neural networks can be used for sequences or spectra. For spectra, they are binned and if there is a peak within a bin the intensity of that peak or a binary indication of peak presence can be used. For sequences they are one hot encoded with each character getting its own channel for input to the CNN. In the case of sequences, early layers in the CNN can learn sequence motifs that are important for predicting the target value.

Optuna can be used to optimize or tune hyperparameters for a neural network, both those related to the neural network architecture and the training hyperparameters such as learning rate. Optuna is more efficient than GridSearchCV because it tests hyperparameters that are likely to be more successful. Optuna offers multiple choices for samplers (the part that chooses which combinations of hyperparameters to test) and pruners (which decide which trials to abandon before training has completed). We will use the TPESampler (Tree-structure Parzen Estimator) which begins by randomly testing a hyperparameters, splitting trials into good and bad trials based on some percentile threshold, then calculates the probability distribution for hyperparameters for each group. The sampler will then test hyperparameters with high probability of being in the good group and low probability of being in the bad group. This is a type of Bayesian optimization. For pruner we will use the MedianPruner which prunes trials that are performing worse than the median of previous trials at equivalent epochs after a set number of warmup epochs.

Chemistry example code: https://colab.research.google.com/github/aswalker-lab/AIChemBiochem/blob/main/8_Convolutional_neural_networks/8_Chemistry_Convolutional_Neural_Network.ipynb
Biochemistry example code: https://colab.research.google.com/github/aswalker-lab/AIChemBiochem/blob/main/8_Convolutional_neural_networks/8_Biochemistry_Convolutional_Neural_Network.ipynb