3 ms·

So what's the activation function on the last layer? Linear? With the intriguing exception of the Global Temporal Pooling layer, this matches up with a lot of

• benanne 12y ago
  Yep, it's linear. This is essentially a regression task and the distribution of the factors across the dataset is pretty close to Gaussian in most cases, so it made sense not to have any nonlinearities there. As a sidenote, if it weren't for the L2-pooling in the global temporal pooling layer, the network would be completely piecewise linear from input to output :)