Yea, this paper says that the embarrassingly simple linear is better than transformers for time-series: https://arxiv.org/abs/2205.13504 https://arxiv.org/abs/

Yea, this paper says that the embarrassingly simple linear is better than transformers for time-series: https://arxiv.org/abs/2205.13504 https://arxiv.org/abs/2205.13504 from the abstract: "Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss..."