4 ms·

We (I included) have been doing this EXACT thing for a long with with the stock market. We take ALL the data, build a model, then ask for an output buy/sell sig

We (I included) have been doing this EXACT thing for a long with with the stock market. We take ALL the data, build a model, then ask for an output buy/sell signal. The idea for using this type of model, from quants, was applied to language, THAT was the new idea. Now it sounds like you are saying "what if" we took a step backwards, but again, we this is exactly how we use these models now, is trained on data.

binarymax 3y ago
Sorry, but are you suggesting that transformers have been used by quants prior to the language application? LLM these days implies a transformer architecture.
- kylebenzle 3y ago
  Yes [1] (1) https://arxiv.org/pdf/2208.08300.pdf https://arxiv.org/pdf/2208.08300.pdf Aug 2022, TRANSFORMER-BASED DEEP LEARNING MODEL FOR STOCK PRICE PREDICTION International Journal of Computational Intelligence and Applications Something similar was an exercise/project in the NYCDSA "Data Science Boot Camp" I did nearly 2 years ago. I am fairly sure that applying these models to language was secondary, right?
  - shwaj 3y ago
    The transformer model was introduced in the 2017 paper “Attention is all you need”, which applied it to machine translation of human language.
  - claytonjy 3y ago
    No, language models were not a secondary application transformers. From the abstract of the paper you posted: > The transformer model has been widely leveraged for natural language processing and computer vision tasks, but, to the best of our knowledge, has never been used for stock price prediction task at DSE. In general, all the fancy DL stuff from the last decade started on either images (e.g. CNNs) or text (e.g. RNNs, transformers). Applying them to tabular data is generally an afterthought, and for time series in particular is rarely worth the time and compute. The paper you posted is not very good and should not be used to inform your own work. It includes no comparisons to baseline models, and even the abstract describes the results as "acceptable".
all2 3y ago
One question before I spend 50k training a model: but does it work? IE, does the income yield outpace the cost of training? I'm assuming that the model needs to be retrained at some frequency, though I'm guessing some of that retraining would be model fine tuning... could all of the training be continual fine tuning? Would there be any benefit by training from scratch?
- claytonjy 3y ago
  Almost certainly not worth it on anything but an institutional scale, and even then most efforts fail or are huge wastes of time and money. Stick to good data and simple techniques, and even then be aware you're extraordinarily unlikely to post higher gains than you'd get with DCA and index funds.

binarymax 3y ago

Sorry, but are you suggesting that transformers have been used by quants prior to the language application? LLM these days implies a transformer architecture.

kylebenzle 3y ago
Yes [1] (1) https://arxiv.org/pdf/2208.08300.pdf https://arxiv.org/pdf/2208.08300.pdf Aug 2022, TRANSFORMER-BASED DEEP LEARNING MODEL FOR STOCK PRICE PREDICTION International Journal of Computational Intelligence and Applications Something similar was an exercise/project in the NYCDSA "Data Science Boot Camp" I did nearly 2 years ago. I am fairly sure that applying these models to language was secondary, right?
- shwaj 3y ago
  The transformer model was introduced in the 2017 paper “Attention is all you need”, which applied it to machine translation of human language.
- claytonjy 3y ago
  No, language models were not a secondary application transformers. From the abstract of the paper you posted: > The transformer model has been widely leveraged for natural language processing and computer vision tasks, but, to the best of our knowledge, has never been used for stock price prediction task at DSE. In general, all the fancy DL stuff from the last decade started on either images (e.g. CNNs) or text (e.g. RNNs, transformers). Applying them to tabular data is generally an afterthought, and for time series in particular is rarely worth the time and compute. The paper you posted is not very good and should not be used to inform your own work. It includes no comparisons to baseline models, and even the abstract describes the results as "acceptable".

all2 3y ago

One question before I spend 50k training a model: but does it work? IE, does the income yield outpace the cost of training? I'm assuming that the model needs to be retrained at some frequency, though I'm guessing some of that retraining would be model fine tuning... could all of the training be continual fine tuning? Would there be any benefit by training from scratch?

claytonjy 3y ago
Almost certainly not worth it on anything but an institutional scale, and even then most efforts fail or are huge wastes of time and money. Stick to good data and simple techniques, and even then be aware you're extraordinarily unlikely to post higher gains than you'd get with DCA and index funds.