Dilated Causal CNNs - Avesta-Ahmadi/Univariate-Time-Series-Forecasting GitHub Wiki

Convolution neural networks are extended in two ways: using causal convolution and using dilated convolutions. Causal convolutions are introduced in order to:

  • Assure the model cannot violate the ordering in which we model the data.
  • Prediction at timestep t cannot depend on any future timesteps.

Here is attached the schematic of a causal convolution network.

NNfig20

Source: Oord, et al. WaveNet: A Generative Model for Raw Audio. ArXiv.org, 19 Sept. 2016, arxiv.org/abs/1609.03499.

Dilated convolution:

  • Dilated convolution is a convolution where the filter is applied over an area greater than its length by skipping some input values with a certain step.
  • Dilated convolutions are used to increase the receptive field by orders of magnitude, without significantly increasing computational cost.

NN_fig21

Source: Oord, et al. WaveNet: A Generative Model for Raw Audio. ArXiv.org, 19 Sept. 2016, arxiv.org/abs/1609.03499.

The concept of dilated causal convolutions will be used in the wavenet and seriesnet structure.