3 ms·

ChatGPT Trading Algorithm Delivers 500% Returns in Stock Market

13 points by mgh2 3y ago

rafamvc 3y ago
The timeframe was only 1 year, therefore this is a rather inconclusive test. It warrants more investigation, but if a hedge fund understands that feeding good news or bad news to an AI makes the stock market move, eventually they will control the news factory to control the AI. It is a neat trick though. I can't read that fast and I can't make 500%.
- emmender 3y ago
  the stock market for short-term traders is a zero-sum game. if there is a chatgpt trick that produced alpha, it would be adopted by every hedge-fund reducing its effectiveness (and alpha generation). so, these sorts of tricks depend on suckering retail traders who as a group are the losers. but the stimmies have run out, and so will these tricks.
jonex 3y ago
Interesting experiment and a good start at evaluating the concept, in particular, the fact that they compared to other methods makes it way stronger than similar concepts that forget to even compare to index. However, even though they clearly tried to avoid obvious pitfalls of backtesting, they also left lots of potential sources for bias. Their method of collecting news is based on what popped up in their post-facto web search, it also relied on them classifying them for relevance. This can generate bias compared to, for instance, feeding all articles tweeted from a specific news source irregardless of "relevance". I also wonder what assumptions they've made about timing, slight errors in timing estimated could have huge impact on results. The fact that the shorting seemed to make for the most part of the gains (although the long strategy was not terrible at 50% gain vs index going down) might also be something somewhat biased towards the fact that this was a downturn year and it might not fare as well in a more regular year.
- emmender 3y ago
  ChatGPT is hot news. Professor wants in on the action. He and his grad-student come up with a thesis. Then they cherry-pick evidence supporting the thesis and concoct a catchy headline that makes it to hacker news.
kolinko 3y ago
They tested six different strategies and only two-three of the strategies returned profit on this year. Even the ones that turned out profitable were only profitable within the first half of the year and provided no return during the second half of the year. Seems similar to p-hacking.
- adsgnauionio 3y ago
  Let's imagine that they started by investing equally in each of their seven strategies. Three of them lost money, but the other four made colossal returns. Eyeballing the chart, this combined strategy would have made at least 12/7 times returns, which is a 71% increase in a period when the market as a whole lost 12%. The results are implausibly good, period.
adsgnauionio 3y ago
>The researchers collated 67,586 headlines pertaining to 4,138 unique companies during this time, using web scraping techniques. These headlines were subsequently filtered for relevance, with the team narrowing their focus to full articles and press releases, excluding any stock-gain or stock-loss headlines. They also removed duplicate news to ensure only fresh information was used. 67,586 headlines over more than a one-year period is under 200 headlines per day. One human analyst could trivially perform sentiment analysis on that many headlines before lunch. Practically perfect sentiment analysis on this scale is thousands of years old. There is no "special sauce" from ChatGPT.
deleted 3y ago
[deleted]
known 3y ago
[dead]

13 points by mgh2 3y ago

The timeframe was only 1 year, therefore this is a rather inconclusive test. It warrants more investigation, but if a hedge fund understands that feeding good news or bad news to an AI makes the stock market move, eventually they will control the news factory to control the AI. It is a neat trick though. I can't read that fast and I can't make 500%.

emmender 3y ago
the stock market for short-term traders is a zero-sum game. if there is a chatgpt trick that produced alpha, it would be adopted by every hedge-fund reducing its effectiveness (and alpha generation). so, these sorts of tricks depend on suckering retail traders who as a group are the losers. but the stimmies have run out, and so will these tricks.

jonex 3y ago

Interesting experiment and a good start at evaluating the concept, in particular, the fact that they compared to other methods makes it way stronger than similar concepts that forget to even compare to index. However, even though they clearly tried to avoid obvious pitfalls of backtesting, they also left lots of potential sources for bias. Their method of collecting news is based on what popped up in their post-facto web search, it also relied on them classifying them for relevance. This can generate bias compared to, for instance, feeding all articles tweeted from a specific news source irregardless of "relevance". I also wonder what assumptions they've made about timing, slight errors in timing estimated could have huge impact on results. The fact that the shorting seemed to make for the most part of the gains (although the long strategy was not terrible at 50% gain vs index going down) might also be something somewhat biased towards the fact that this was a downturn year and it might not fare as well in a more regular year.

emmender 3y ago
ChatGPT is hot news. Professor wants in on the action. He and his grad-student come up with a thesis. Then they cherry-pick evidence supporting the thesis and concoct a catchy headline that makes it to hacker news.

kolinko 3y ago

They tested six different strategies and only two-three of the strategies returned profit on this year. Even the ones that turned out profitable were only profitable within the first half of the year and provided no return during the second half of the year. Seems similar to p-hacking.

adsgnauionio 3y ago
Let's imagine that they started by investing equally in each of their seven strategies. Three of them lost money, but the other four made colossal returns. Eyeballing the chart, this combined strategy would have made at least 12/7 times returns, which is a 71% increase in a period when the market as a whole lost 12%. The results are implausibly good, period.

adsgnauionio 3y ago

>The researchers collated 67,586 headlines pertaining to 4,138 unique companies during this time, using web scraping techniques. These headlines were subsequently filtered for relevance, with the team narrowing their focus to full articles and press releases, excluding any stock-gain or stock-loss headlines. They also removed duplicate news to ensure only fresh information was used. 67,586 headlines over more than a one-year period is under 200 headlines per day. One human analyst could trivially perform sentiment analysis on that many headlines before lunch. Practically perfect sentiment analysis on this scale is thousands of years old. There is no "special sauce" from ChatGPT.

deleted 3y ago

[deleted]

known 3y ago

[dead]