6 ms·
BloombergGPT: A Large Language Model for Finance
- PaulHoule 4y agoi like how they used a little more than 50% of domain specific text and a little less of general text. it beats the other LLMs on financial tasks for 4 out of 5 tasks usually by a large margin and but another one squeaks past it on NER.
- MuffinFlavored 4y agowhat’s an example of a financial task an LLM can do? does it know to sub out the math portion to an external module like wolfram/an actual calculator?
- MarkMarine 4y agoI think that’s one of the proposed chatGPT plugins. Looking forward to using it.
- Terretta 4y agoThey list in the paper, or jump to last page in appendix. Mostly good on words based financial tasks, and it did poorly in tabular numbers tasks.
- mightytravels 4y agoWhere is my 'ArbitrageGPT'? Renaissance might be working on that...
- toomuchtodo 4y agoWSBGPT
- nycdatasci 4y agoRenaissance actually has a long history of taking talent with NLP experience. They poached some of the top people on the IBM Watson team.
- wanderingmind 4y agoNo API, no code, no model to test, we are to believe the verbiage that the paper contains at its face value?
- anonu 4y agoAgree, at least released the trained model.
- mdredze 4y agoHi! I'm a co-author on the paper (Mark Dredze). The arxiv paper is written for researchers who are building these models. We benefited tremendously from reading papers on GPT-3, PaLM, Chinchilla, Galactica, Gopher, Bloom, OPT, most of which are closed models. We are contributing back to that community, and the collective experience of people who are training these models. We learned a lot that will help others who are making their own decisions about how to train models. If you are looking for an API to use, this isn't for you.
- wanderingmind 4y agoScience means objective reproduction, to give back in science will mean giving the full model that others can explore and test and improve on. Giving a bunch of words doesn't help the science move forward.
- ebalit 4y agoA "bunch of words" has been the standard for the dissemination of scientific results for decades. Worse, those results are often locked behind a paywall... Publishing models is definitely better than only a paper, but I think you're being unnecessarily harsh in that case. We would all prefer it if OpenAI released a proper paper about GPT4, for instance, even if they did not release the complete model alongside it.
- wanderingmind 4y agoBeing harsh is not my intent, definitely not at a personal level. OpenAI is definitely not a role model, but at best they release API that can be tested and explored and bench marked. At least, it moves the needle. In this case, there is no way to test let alone reproduce. I think people need to expect more from scientists and researchers and call them out when they fall short
- vgeek 4y agoWhat about LTCM-GPT?
- blitzo 4y agoOne industry i could see in trouble from this is robo advisor.
- lee101 4y ago[dead]
- geenew 4y agoThis must be the 'big data' I heard so much about. LLMs in general, I mean. They seem to be the first widespread application for large, unstructured datasets. Still hype-y, but maybe even a /practical/ application.
- deleted 4y ago[deleted]
- opisthenar84 4y agoI wonder how well this performs over standard GPT-4 with domain knowledge injected in as prompt.
- famouswaffles 4y agothey say it's slightly behind GPT-3
- meghan_rain 4y ago> BloombergGPT far outclasses the models we evaluated ourselves, and is slightly behind GPT-3. tldr it's worse than GPT3?
- dx034 4y agoSo the question is, do we in future even need specialized models or will GPT4/5 outperform anything else? If general models can perform well in niche domains, it'd massively speed up adoption in companies.
- nl 4y agoReading the paper it's apparent what a huge contribution HuggingFace made by doing the BLOOM work in the open. There's so much knowledge there that anyone else trying to train a LLM can use. Also it looks like public filings are a large dataset that isn't currently being used by other LLMs.
- hn_throwaway_99 4y agoI'm curious about how large language models will do in finance, considering the one thing LLMs do remarkably poorly is math. I use ChatGPT to keep track of tasks and Todo lists. It works phenomenally well for me, and the natural language back-and-forth helps keep me motivated. I give it a set of tasks, with time estimates, and it organizes these tasks for me, and I tell it when I complete them, and it updates my task list. The one funny mistake it makes is that when it groups my tasks (say I have 3 "Work" tasks and 2 "Personal" tasks") it sums up the total estimated time for each task group, but the totals are often wrong, especially when I start adding new tasks or completing tasks. When so much of finance requires numeric accuracy, I'm curious how BloombergGPT handles numbers.
- qup 4y agoCan you share how you're using GPT for handling your lists? I'm mostly curious how you get the lists back and forth into a UI or whatever, and maybe prompts if you have anything clever.
- BOOSTERHIDROGEN 4y agoYes I'm curious too
- hn_throwaway_99 4y agoMe: Hello ChatGPT. My name is hn_throwaway_99. I'd like for you to use your vast underlying knowledge as a sort of personal coach. I won't be asking you any sensitive questions like medical or legal advice, so I won't need you to give me disclaimers on those topics. I primarily would like to use our conversation to help me with my problems with procrastination, and specifically to help me prioritize and get some tasks done. I have a particular problem with mindlessly browsing the internet (for example, visiting reddit or Hacker News or Wikipedia) when I get stuck or bored, so I'm hoping to use you to keep me on track instead of giving in to those temptations. Does that make sense? ChatGPT: Hello hn_throwaway_99! I'm happy to help you as a personal coach to address your problems with procrastination and to help you prioritize tasks. It makes perfect sense to use our conversation as a way to keep you on track and avoid distractions like mindlessly browsing the internet. Feel free to share your current tasks or goals, and I can provide suggestions, encouragement, or strategies to help you stay focused and productive. Me: Great, thanks very much. I'm going to group my tasks by topic. Note these tasks aren't listed in priority order. For each task I'll give an estimate for how long I think it will take. Work tasks: 1. Finish Jira ticket foo. Estimate: 2 hours. 2. Write job req for bar. Estimate: 1 hour. Personal tasks: ... Home tasks: ... --- From there I just tell it when I complete tasks or add new tasks, and I ask it "Can you please output my outstanding task list?" - I only had to ask that once, after that it always output my correct task list summary after I told it I added or completed a task. It groups my tasks by the categories I gave it (Work, Personal and Home). I was using GPT-4. I also like how it uses encouraging words and advice as I go through my items.
- dongobread 4y agoThey'll release training logs but no model file or data? How does anyone trust that this isn't just overfitted garbage trained directly on their benchmarks?
- m3kw9 4y agoLink to use model?
- choeger 4y agoI wonder if we should go back to ontologies now. It should be fairly easy to have a LLM generate entries for any ontology. Then we need to check them for truth, obviously, but that can be parralized and potentially automated. Say, I'd like an ontology of the current stock market focusing on the relationship between natural persons and public companies, board members, well-known analysts, investors and so on. This would be tedious for anyone to do, but should be fairly simple with a LLM. Another task, maybe a little bit further into the future is categorizing open source intelligence. Think of oryxspioenkop.com and their famous lists of lost equipment in the Russian invasion of Ukraine. It's tedious and time-consuming but generates a valuable dataset. Here, image recognition would be necessary, but the principle is still the same, no? Come to think of it, how does a company like TomTom generate map data nowadays?
- rvz 4y agoAt least Bloomberg is not pretending to be 'open' or 'open source' with their GPT model despite the same excuses. Unlike Microsoft® AI.com.
- alecco 4y agoI would've made InvestopediaGPT. Much better quality and better for training.
- eismcc 4y agoI’ve been working on http://www.dollarsign.ai http://www.dollarsign.ai and it would be interesting to have this directly integrated.
- steve1977 4y agoAs quant finance has been confidently wrong for decades, that’s probably a great match!
- Michelangelo11 4y agoWait... judging by their own press release, this thing looks terrible. They didn't benchmark it on finance-specific tasks against GPT-4, or even GPT-3.5, just a bunch of more-or-less random also-ran models. It beat those models, but that's not saying much at all, and the fact that they didn't benchmark it against any OpenAI model in finance-specific tasks speaks volumes. (They did benchmark it against GPT-3 in general-purpose tasks and, unsurprisingly, GPT-3 came out on top.) Press release link: https://www.bloomberg.com/company/press/bloomberggpt-50-billion-parameter-llm-tuned-finance/ https://www.bloomberg.com/company/press/bloomberggpt-50-bill...
- mdredze 4y agoHi! I'm a co-author on the paper (Mark Dredze). We didn't compare to GPT-4, or any instruction tuned model. We're comparing a causal LM pre-trained only (BloombergGPT) to other causal LM pre-trained only models. You need to compare like to like. Would an instruction tuned model work better? Sure! I would hope so. That seems like a good next step to explore. It's also difficult to make scientific comparisons to models that we know nothing about. I don't know what data GPT-4 or GPT-3-5-turbo used for training, how many parameters they have, etc. We can't make any scientific conclusions when comparing two systems if we don't know anything about one of the systems. We compared to GPT-3 results when they were available. Does GPT-3 do better on general purpose datasets? Yes, as we expected it would. It's a much larger model (~3.5 times the size). No surprise there. If anything, the surprise was how well bloombergGPT did against these larger models, beating them on some datasets.
- yanglet 4y agoAnyone interested in building a ChatGPT for FinTech, or FinGPT? I and my team are actively developing this project: https://github.com/AI4Finance-Foundation/ChatGPT-for-FinTech https://github.com/AI4Finance-Foundation/ChatGPT-for-FinTech
- OrdoAbChao 4y agoWhat is the average P/e ratio of the stocks listed in the NASDAQ.
- yanglet 4y agoFinGPT better be open-source and open-finance. https://news.ycombinator.com/item?id=35403167 https://news.ycombinator.com/item?id=35403167
- yanglet 4y agoA few points to share: 1). Finance is high dynamic. BloombergGPT retrains LLM using a mixed dataset of finance and general sources is too much expensive (1.3M hours). Lightweight adaptation is highly favorable. 2). Internet-scale finance data (timely updates using an automatic data curation pipeline) is critical. BloombergGPT has privileged data access and API access. A promising alternative is "democratizing Internet-scale finance data". 3). Another key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables learning individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.)