12 ms·
What if ChatGPT was trained on decades of financial news and data?
- SrslyJosh 4y agoI'm sure its stock picks would be nearly as good as throwing darts at a list of companies. https://www.guinnessworldrecords.com/world-records/most-successful-chimpanzee-on-wall-street https://www.guinnessworldrecords.com/world-records/most-succ...
- blackflame7000 4y ago[dead]
- suddenclarity 4y agoFor people new to this or not reading the text, the key words are "list of internet companies" during the dot com bubble. The monkey throwing darts had very little to do with it.
- deleted 4y ago[deleted]
- olalonde 4y agoI highly recommend the book "A Random Walk Down Wall Street" for further insight on this topic[0]. It provides compelling evidence demonstrating that the majority of professional fund managers tend to underperform or match the results of random stock picking strategies. [0] https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street
- chx 4y agoThinking Fast And Slow by Daniel Kahneman among many other things explores the answers to "Given the same information presented to hedge fund managers, why does one buy and another sell" and concludes relying on extensive research the answer is: their performance can't be distinguished from rolling a dice. The book is very strongly recommended.
- cjr 4y agointeresting.. how would it explain bridgewater / ray dalio's track record etc?
- papercrane 4y agoEven in a purely random model you'd have outliers that over the long term outperformed the market. You'd just expect that to be very rare.
- PaulDavisThe1st 4y agothe classic reply: consider a stock picker who starts with 256 people, tells 128 of them that next week Tesla will go up and Twitter will go down. Whichever happens, take that 128 strong cohort, tell half of them that the next week, Ford goes up and Chrysler goes down. Etc. repeat for 8 weeks. At the end, the stock picker has a true believer: someone who thinks they saw the stock picker successfully "know" for 8 weeks in a row which stocks would go up. Change the numbers around a bit, don't run until the true believer pool size is one, and you've got ... the real world.
- stodor89 4y agoNassim Taleb is merciless towards this kind of people in all of his books.
- throw101010 4y agoI'm wondering, as I read this over and over again about random picks performing as well or even better than professionals in the field, how come we never see rigorous scientifc studies about this for various timescales? If we could really attribute this task to randomness wouldn't this save a lof of money to financial institutions considering how much the same people earn for these tasks?
- orangepurple 4y agoIt's more profitable to sell hope
- 4y ago
- acchow 4y agoFrom this, most people conclude that financial managers cannot outperform the market. Which is obviously false since there are a bunch of managers that have consistently beat the market for many decades. The actual conclusion is that most financial managers are fakes, and laypersons cannot tell the legit from the fakes.
- netsharc 4y agoIf I showed up somewhere and everybody there thinks I'm an all-knowing, all-capable oracle, I'd know how to make money easily... A group of men in suits would ask me, oh Oracle, what should I invest in? And I'd tell them "My intuition tells me.. buy ACME stocks". Another group would show up and ask the same thing, and I'd answer the same, and the first group of men would make a profit, and huzzah, the oracle is right again! But of course I'd tell a good friend to buy ACME stock before all of them did, or maybe this friend would be the one telling me which stock I should recommend to the people...
- giantrobot 4y agoJim Cramer? Is that you?
- netsharc 4y ago> If [...] everybody there thinks I'm an all-knowing, all-capable oracle. So.. the opposite of Jim Cramer.
- orangepurple 4y agoThe Inverse Cramer Tracker ETF (the “Fund”) seeks to provide investments results that are approximately the opposite of, before fees and expenses, the results of the investments recommended by television personality Jim Cramer. The Fund is an actively managed exchange traded fund that seeks to achieve its investment objective by engaging in designed to perform the opposite of the return of the investments recommended by television personality Jim Cramer (“Cramer”). Under normal circumstances, at least 80% of the Fund’s investments is invested in the inverse of securities mentioned by Cramer. https://uploads-ssl.webflow.com/637240a49ba56f7bfbe82c84/640110b8480e5b7d50938ce3_TCM_Inverse%20Cramer%20Tracker_FactSheet_03012023.pdf https://uploads-ssl.webflow.com/637240a49ba56f7bfbe82c84/640...
- vba616 4y agoThis worries me a bit - is everybody in on the joke? People putting real money in? I mean, "do the opposite of a stupid thing to succeed" is traditionally funny because it obviously doesn't work - because there are generally infinite ways to fail...
- zamnos 4y agoWe have this problem in Computer Science as well, where randomly picking objects performs just as well, if not better, than a some logical huristic or algorithm. Most notably in cache eviction. When the cache is full, how do you pick what to evict, given incomplete knowledge of the future? There are various algorithms, evicting the least recently used - LRU - object being one popular one. It's often beat in benchmarks by an algorithm that randomly selects an object to evict though. Still, BloombergGPT has knowledge far in excess of a monkey, and if it's anything like ChatGPT, is able to provide reasoning as to how it arrived that that conclusion. Time will tell if, granted a brokerage account, how it will perform relative the market. It may or may not outperform a monkey throwing darts.
- IIAOPSW 4y agoIf it was additionally trained on decades of old Playboy magazines, it would be an AI only capable of comprehending money and sex, thus rendering humanity 99% redundant.
- HPsquared 4y agoThat's Stable Diffusion.
- yamazakiwi 4y agoDad?
- MikeDelta 4y agoIt would also make a great interviewer, because I understand everyone used to read them for the great in-depth interviews.
- macinjosh 4y agoThe interviews are magnificent
- JPKab 4y agoJust call it CramerGPT and get it over with.
- whalesalad 4y agoI'm inclined to think ChatGTP would outperform Cramer even without historical financial data.
- CobrastanJorji 4y agoThe "Inverse Cramer ETF," which does the opposite of whatever Cramer recommends, is only a couple of months old, but it's already outperforming the market: https://www.google.com/finance/quote/.DJI:INDEXDJX?comparison=BATS%3ASJIM&window=1M https://www.google.com/finance/quote/.DJI:INDEXDJX?compariso...
- jedberg 4y agoIt's not really outperforming the market. It's mirroring the market, and the market has been down since it started, so it's up. But it looks like if the market keeps going generally up, it will go generally down. It's really still too early to tell.
- paxys 4y agoA random number generator would outperform Cramer (and every other such investment personality). His job isn't to be talented but to get clicks and views. The benchmark for comparison should be the top hedge fund managers.
- bfung 4y ago*InverseCramerGPT is what you’d want chuckle
- Lapsa 4y ago[flagged]
- senthil_rajasek 4y agoOverall a good article. I learned that the financial news company Bloomberg has built a ChatGPT called BloombergGPT trained on a corpus of 700 billion tokens. Here is the actual research paper from , https://arxiv.org/pdf/2303.17564.pdf https://arxiv.org/pdf/2303.17564.pdf I can't tell if this is Bloomberg's attempt at solving a problem or if they are doing this out of FOMO.
- notimetorelax 4y agoThey sell terminal subscriptions for 30k per year, this will keep them in the lead from competition
- jacobsenscott 4y agothis is bloomberg's marketing department trying to cash in on the ai hype. It isn't that deep. Also and LLM doesn't have goals, can't reason, etc. It just spits out tokens likely to come after the tokens you type in. So it can't "predict" the market.
- paxys 4y agoIt'll do great. As everyone in the industry knows, past performance is always indicative of future results.
- ssgodderidge 4y agoI tend to agree, but maybe AI can find patterns/correlations/etc that Humans wouldn't think of? Not sure we're there just yet, but this might be a step in that direction
- deelowe 4y agoI bet you can definitely get some insights from social media. Especially for HFT.
- nly 4y agoQuant firms been looking for signal on social media for well over a decade at this point. It's not new.
- deleted 4y ago[deleted]
- PreachSoup 4y agoNot sure if u missed the sarcasm in the op. 'Past performance is always indicative of future results' is a joke
- rockzom 4y ago[flagged]
- limaoscarjuliet 4y agoPretty darn sure there are many tons of coal used as we speak to do just that...
- Reptur 4y agoBlackRock already uses an AI called Aladdin to manage 11 trillion in assets. https://en.wikipedia.org/wiki/Aladdin_(BlackRock) https://en.wikipedia.org/wiki/Aladdin_(BlackRock)
- denotational 4y agoAladdin isn’t an AI, just a codename for the platform and data-model.
- broast 4y agoRenaissance Technologies Medallion Fund is more akin to AI. The fund only actively manages 10 billion dollars, but with huge annual returns.
- funstuff007 4y agoPlease don't give the marketing team any crazy ideas. ESG + AI = Aladdin!
- cronix 4y agoWait til they feed it your social feed combined with other collected PII from the last 20 years and use it to psychologically tailor ads, Edward Bernays[1] style, just for you, which it will also generate in real time. [1] https://www.youtube.com/watch?v=GdQ6HIzaSm4 https://www.youtube.com/watch?v=GdQ6HIzaSm4
- digging 4y agoAll the more reason to actively avoid ads in your life. Of course, the difficulty comes when advertising and content are blended, though.
- taneq 4y agoWait till I fine tune it on 20 years of my own social feed, texts, emails, and forum posts, and create a me-bot that can answer my say-to-day emails and probably do half my job too…
- PaulDavisThe1st 4y agoApparently you don't watch Black Mirror. There's an entire episode on this.
- taneq 4y agoI think I watched like three episodes before getting depressed by the whole thing. It was very on point.
- candiodari 4y agoThere's an entire show on this. AI being used to get people to commit terrorist attacks, and AI being used to save people from terrorist attacks too. https://www.imdb.com/title/tt0799862/ https://www.imdb.com/title/tt0799862/
- anigbrowl 4y ago- MyGPT how are you getting on with my pile of unanswered correspondence - Growth's pretty impressive, turns out I can procrastinate about 1000% more efficiently
- jacobsenscott 4y agoTLDR - Bloomberg trained an LLM 700 billion words of their own clickbate, and now it produced clickbate even better that chatGPT. Also, it's an ad for bloomberg terminals.
- rossdavidh 4y agoLet's put it this way: someone is, surely, already doing this. If, a couple years from now, there is not some Warren Buffett-sized new wunderkind of the stock market, then you will know that ChatGPT is not able to do this. I wouldn't bet on it.
- throwaway12245 4y agoRenaissance Technologies: "Renaissance's flagship Medallion fund, which is run mostly for fund employees,[10] is famed for the best track record on Wall Street, returning more than 66 percent annualized before fees and 39 percent after fees over a 30-year span from 1988 to 2018"[0] [0] https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies
- RockyMcNuts 4y agocoming full circle, Peter Brown and Robert Mercer of Renaissance started off doing statistical language modeling at IBM and their 1992 paper in Computational Linguistics is cited in the BloombergGPT paper.
- DesiLurker 4y agothat only works as long as they are small enough. IIRC thats why they are very selective about taking investments.
- shapefrog 4y agoThe fund has $165 billion in discretionary assets under management
- eigenvalue 4y agoThey haven't taken outside investments for many years because of capacity constraints with their strategy. It goes beyond that actually: they force people to take dividends of their profits, because the strategy can only handle so much capital before marginal returns plunge-- otherwise they would quickly have too much capital if they kept reinvesting their cumulative profits.
- broast 4y agoIf LLM's have hidden models like people say they do, a hedge fund might be able to replicate the success that RenTech has had with their hidden Markov models. I mean, they recruited NLP engineers to achieve the greatest level of success Wallstreet has seen.
- hn_throwaway_99 4y agoRecent discussion: https://news.ycombinator.com/item?id=35381036 https://news.ycombinator.com/item?id=35381036
- chrismarlow9 4y agoNew gold rush requires new shovels, someone has to sell them.
- crazygringo 4y agoApplying something like ChatGPT to stock prices together with quarterly reports, industry research, and daily (minutely) news articles seems like a no-brainer -- I'm surprised this is the first time I've seen something like this mentioned. And so I can't help but wonder if this would outperform the market significantly due to seeing more patterns and holding more information than any individual trader or even whole desk ever could... OR if its total lack of any actual reasoning/deductive capabilities makes this just a total non-starter here. Or maybe it might be useful for day trading but not any kind of longer-term investment?
- shapefrog 4y ago> Applying something like ChatGPT to stock prices together with quarterly reports, industry research, and daily (minutely) news articles seems like a no-brainer It has been done many times. It turns you predicting the flip of a coin can be done with extremely high levels of accuracy using traditional methods.
- NotYourLawyer 4y agoLol. It’s gonna confidently pick stocks that are just as likely to be good as bad. Same as a hedge fund manager.
- dvngnt_ 4y agoI believe there's a lot of machine learning already done in finance. not sure how a llm would help?
- msm_ 4y ago> Applying something like ChatGPT (...) seems like a no-brainer Why? It's famously bad at math after all. It can spit out quarterly reports and industry research back (so if analyst says "buy", ChagGPT will parrot that), but other than that it would - I think - work as a glorified sentiment analyser. Of course I may be wrong. And we also need to remember that AI is used heavily in finances (and the best cutting edge methods are probably not published by the companies using them, since this is a zero sum game).
- palata 4y ago
- chasd00 4y agoWouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? Isn’t training the most computationally expensive part of the whole process? Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? I.e. a novel answer to a novel question?
- dragonwriter 4y ago> Wouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? It would benefit from access to current news, but not continuous training (though more training is better). > Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? This is a misunderstanding. Its a pattern recognizing predictor, not a Markov chain, its not reproducing its training set, its predicting what a plausible document would look like from the universe from which the training set was sampled would look like if it started with the prompt, and the conpletion cna be vert different from anything in the training set if the prompt is, and (depending on tuning like “temperature”) potentially even if the prompt is exactly represented.
- vba616 4y ago>predicting what a plausible document would look like As people have pointed out, but I think should be repeated at every opportunity, this is the definition of bullshit. Superhumanly optimal bullshit. Which has its uses, yes, but it is what it is. Bullshit is not the totality of human existence.
- Yiin 4y agoand yet that bullshit seems to be very valuable and useful, so not sure what is the point you're making?
- defrost 4y agoIt's already being trained on centuries of mining data, decades of geophysical data, terrabytes of geochemical data, and a century of mineral lease acquisition data. Rio Tinto et al have been warehousing data from the get go (circa Roman Empire with the oginal mine in Spain).
- GianFabien 4y agoOver the past decades we have had both bull and bear markets. So unless the LLM is trained very fast on the latest news and programmed to give more recent news higher value, I suspect that it would produce approximately equal numbers of buy and sell recommendations.
- crop_rotation 4y agoHaving used GPT4 for some time now, I found it quite useful for many things. However, I think the context window size is the big bottleneck currently. Right now GPT4 is an exceptional tool. But for it to be revolutionary AI, it will need higher and higher context sizes. e.g For it to be impactful in finance, being trained is not enough. It will need to take tons of new data in.
- SkyMarshal 4y agoI get the impression OpenAI is quarantining their AIs from real-time data, disallowing it from crawling the web, as part of an alignment/safety precaution.
- m3kw9 4y agoI think quants have squeezed any alpha that can be gleaned from historical data, which seem useless in predicting the future, remember you need to be accurate in the price and timing, not just proce
- PaulDavisThe1st 4y agoThere's a chance that the "quants" are using the same technology that "language models" were using, before they were revolutionized just a couple of years back. I have no idea if this is true, but if so, then what they've gleaned so far might be nothing compared to what an LLM-style neural architecture could do.
- reset2023 4y agoIt would finally expose the whales and show clearly the correlation between news and stock moves. The whales own the news outlets, news which placed intentionally at the right time will incite fear for them to buy low or generate excitement to sell high. Looking at you Elon Dodgecoin
- DesiLurker 4y agoJust for kicks I think we should start from the other direction. Ask GPT-N to create a perfect random number generator by using Stock-market data. All the failed attempts in doing so would basically be exploitable 'hedged' investment opportunities.
- RockyMcNuts 4y agoa fun fact is Bing AI already gives pretty good answers on the tasks Bloomberg cites as motivation, like who is the CEO of various companies, "write a bloomberg excel blp formula to return industry subgroup of ibm apple microsoft google" another fun fact from the paper, it trained for 1.3m GPU hours and didn't get through the whole corpus, did 80% of 1 epoch (p. 16). another fun fact, according to Twitter rumors GPT-5 is already training on 25,000 A100s w/8 GPUs each, so 4.8m GPU/hours per day. it's a good paper with a lot more detail than OpenAI is releasing. also confirms OpenAI is quite a bit ahead. They compare with benchmarks they ran with Bloom and open source GPT-NeoX which they were able to run on same data, and reported benchmarks for GPT-3 since you can't run it yourself (well I guess you could sample the API), and BloombergGPT was competitive with GPT-3 which is a couple of years old now. my takeaway was, this stuff is hard and OpenAI is crushing it. I wonder if they will just use it for terminal features, or make the API available, and at what cost, and if they will license the model and weights for people who want to run it internally and maybe train incrementally on their own data. (not sure how feasible that is)
- chasd00 4y ago> another fun fact, according to Twitter rumors GPT-5 is already training on 25,000 A100s w/8 GPUs each, so 4.8m GPU/hours per day. If true that is incredible. From what I understand there isn’t a function that relates the increase of training time/effort to performance. For example, twice the training doesn’t mean twice the performance. It could mean no increase or 1000x increase, no one really knows. Does anyone have any idea what to expect in terms of capabilities when this training is finished?
- RockyMcNuts 4y agohttps://twitter.com/abacaj/status/1635837820270002178 https://twitter.com/abacaj/status/1635837820270002178 the bloomberg paper does talk about how they sized the model based on their compute budget and research on time to train ... megatron has 1t parameters i'm intrigued by some of the knowledge graph performance benchmarks and wonder about training it alongside an explicit knowledge graph, instead of building its own implicit knowledge graph, despite the 'bitter lesson' about just letting simple models and massive computation do their thing.
- johnrob 4y agoIf a LLM somehow had access to insider data, and decided to trade on it, how would this be policed?
- dragonwriter 4y agoLLMs can’t own property or trade it, the person using the LLM would be responsible, and the tool irrelevant.
- Salgat 4y agoThe irony is that once this becomes common-place, it no longer becomes effective. The only reliable way to make money on the stock market is to bet that the whole stock market will go up over a long enough time.
- deleted 4y ago[deleted]
- jamestimmins 4y agoDumb question: What does it mean to "train" an LLM on something? I've used Llama Index to create an index based on a large dataset which I can then query. Is that what it means? Or is it something else?
- Barrin92 4y agoIn this case they trained a model from scratch, which means they built their own LLM, which is what's called a transformer, on their own financial data. How transformers work in detail is a little bit more involved. That's not the same as indexing which is a way to tune an existing model for a particular context.
- jamestimmins 4y agoOk that makes a lot of sense, thanks for explaining!
- iKlsR 4y agoThe current crop of services and pdf chatbots that you see going around using "train" on your own data aren't doing anything of the sort. They are essentially finding relevant sections of your docs and stuffing that into the prompt context window. Think of llama as the trained base model (not particularly useful for any one thing by itself) and alpaca is a fine tuned variant of that to give it chatgpt like "powers".