3 ms·

LSTMs have been going the way of the dinosaurs since 2018. If you really need a complex neural network (over 1D convolution approaches), transformers are the cu

LSTMs have been going the way of the dinosaurs since 2018. If you really need a complex neural network (over 1D convolution approaches), transformers are the current SOTA. Example implementation in "temporal fusion": https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/stallion.html https://pytorch-forecasting.readthedocs.io/en/stable/tutoria... Mind you, in practice I've found these DL approaches overkill for simple problems of the "trends + cyclics + noise" kind.

nerdponx 4y ago
My impression is that these kinds of models need a lot of data to train properly. I made a comment elsewhere in this thread, musing that tree ensemble models could do the same job, as a kind of low resolution quantized approximation. If you have experience in this area of research, I'd love to hear your thoughts on that.