6 ms·
Sorry, but are you suggesting that transformers have been used by quants prior to the language application? LLM these days implies a transformer architecture.
by binarymax 3y ago
Sorry, but are you suggesting that transformers have been used by quants prior to the language application? LLM these days implies a transformer architecture.
- kylebenzle 3y agoYes [1] (1) https://arxiv.org/pdf/2208.08300.pdf https://arxiv.org/pdf/2208.08300.pdf Aug 2022, TRANSFORMER-BASED DEEP LEARNING MODEL FOR STOCK PRICE PREDICTION International Journal of Computational Intelligence and Applications Something similar was an exercise/project in the NYCDSA "Data Science Boot Camp" I did nearly 2 years ago. I am fairly sure that applying these models to language was secondary, right?
- shwaj 3y agoThe transformer model was introduced in the 2017 paper “Attention is all you need”, which applied it to machine translation of human language.
- claytonjy 3y agoNo, language models were not a secondary application transformers. From the abstract of the paper you posted: > The transformer model has been widely leveraged for natural language processing and computer vision tasks, but, to the best of our knowledge, has never been used for stock price prediction task at DSE. In general, all the fancy DL stuff from the last decade started on either images (e.g. CNNs) or text (e.g. RNNs, transformers). Applying them to tabular data is generally an afterthought, and for time series in particular is rarely worth the time and compute. The paper you posted is not very good and should not be used to inform your own work. It includes no comparisons to baseline models, and even the abstract describes the results as "acceptable".