7 ms·

Reading SEC filings using LLMs

118 points by nnechm 3y ago

nnechm 3y ago
I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the right thing to read, they will do very well on 'reading comprehension'. Open ended writing is more susceptible to errors, especially in questions related to finance. For e.g google's revenues are just as likely to be 280.2 billion vs 279 billion in a probabilistic model that guesses the next part of the sentence - Google's revenues for FY 2022 are .... So this leaves us with the main problem to solve; Serving the right texts to the LLM aka text search. Once the right text is served, we can generate any pretty much anything in the text, Income statements, ceo comments, accounts payable on the fly, For e.g try - `can you get me Nvidia and AMD's income statement from March 2020 ?` as in here. https://imgur.com/gallery/H8Vfd5X https://imgur.com/gallery/H8Vfd5X A few more examples, Apple's sales in China, Google's revenue by quarter: https://imgur.com/a/oCCay3o https://imgur.com/a/oCCay3o Currently, the application supports ~8k companies that are registered with the SEC. Pdfs are still work in progress, so tesla etc don't work as well. The stack is Nextjs on Supabase. So Postgres's inbuilt text search does a lot of heavy lifting. If one thinks of the bigger picture, we can extend/improve this to pdfs and the entire universe of stocks and more. a.k.a a big component of what CapitalIQ, Factset, Bloomberg and Reuters do can now be generated on the fly accurately for a fraction of the cost. Generating graphs with gross margin increasing etc are just one step further and stuff like EV/Ebitda, yet another step further, as one can call a stock pricing api for each date of the report. I would guess a number of LLM applications follow a similar process, ask a question --> LLM converts to query --> datalakes/bases --> searching and serving texts --> answer. Goes without saying, I would appreciate any feedback, especially from those who are building stuff that looks architecturally similar :) !
- paulkon 3y ago
  Could you share more about the prompt you use to get GPT-4 to return a query from the user's question? This is fed into postgres full text search?
  - nnechm 3y ago
    I am using openai's latest functions api, so you can get it to return arguments that will ensure that you get a json, it works pretty well most of the time. The json would then be used to fetch a report from a database.
- anotherhue 3y ago
  I've been trying something similar with parliamentary debates. They're long winded, often full of empty speech, and a chore to read. The LLMs are able to hone in on the details and provide interesting responses like "What questions were asked of the minister that they failed to address" and "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer" Crucially, they're well transcribed: https://api.oireachtas.ie/ https://api.oireachtas.ie/
  - nnechm 3y ago
    I think one thing you can try is to figure out what lies where, so chunking arbitrarily will not work as well as chunking with headings for e.g For e.g, if the question is: "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer." An embedding based search will find it fairly difficult to match this against a text. Based on my experience, you have to figure out what is a nonanswer first (i don't think that's easy but gpt4 is very good at a lot of language based stuff.) you can try Q1: question A1: Answer then prompt GPT, do you think A1 answers the question and then save it. And then, Q1: question, A1: Answer, Q2: Follow up based on the following questions, do you think A1 answers Q1 and then save it to a db. You can then augment it in the code with our own knowledge of how politicians lie, using certain words etc :) to improve what gpt4 might miss...
halotrope 3y ago
This looks great! We have a similar tool on https://markets.sh https://markets.sh but more geared towards news and general information about stocks (You can ask e.g "What are the current issues with the Disney company?") We found the hardest part about this is the pre-ranking of sections and finding the relevant information. Do you feed the whole report into OpenAI?
- nnechm 3y ago
  Thanks, I don't feed the whole report in, unless necessary. Context size is a real limiting factor :), so you need to split it up into chunks that are meaningful on their own and find those relevant to the question. In this case, what are the current issues with the Disney company, This is how I would do it :) I get all the news articles related to Disney, this would then have sections (depends if you are using a json or a scraper), now you would then need to pick only those chunks and the surrounding ones (this would be an on the fly search) and then pass it to openai... This one is a very open ended question, this is what I get in beatandraise.com :) https://imgur.com/a/dqmcrRA https://imgur.com/a/dqmcrRA
  - nnechm 3y ago
    I don't do handle news articles, and was just focusing on filings, so my answer would look for a filing. That's why it looks the way it does :)
refurb 3y ago
I'm pretty sure a primitive version of this has been done for a long time by hedge funds. You basically monitor the SEC website for when filings are made public, quickly parse the document in some automated way and make trades based on the output. It wouldn't surprise me if some hedge fund has been trying to build something with LLMs for past few years (and possibly already deployed it).
- nnechm 3y ago
  You are probably right, but there has been nothing like gpt4 before, having seen some versions before I would wager that their entire application can probably be redone for a fraction of the cost and it would work better... :)
- xcxcx 3y ago
  This was done by hand by people like Footnoted. They received SEC filings using a key red flag word filter into a shared Gmail account with special attention for filings done on Friday night or ahead of the holidays. I think using LLMs to ask a company about their company figures is missing the point. After all, XBRL already exists, and that data is widely accessible anyways. Where LLMs would add more value is analyzing changes in different sections of a company's filings, such as Risk Factors, MD&A, or governance filings.
  - nnechm 3y ago
    Yes, I think the broader point is you have these readers available on call via api, so you can do all that you suggested and more...
- infecto 3y ago
  These already exist and are not primitive. Groups have been applying NLP and analysis against text for longer than you would think.
  - nnechm 3y ago
    Yes, they are definitely not primitive, but considering how the breakthrough in LLMs happened, i.e MSFT, GOOGL were working on it, even as late as 2021, Google's BERT was not really there (at least the one they showed the public in the blog from Google research). So the sudden jump from ok to great thanks to openAI would probably mean a lot of existing systems also need to do that.
sroussey 3y ago
I am working on something similar. I have all the 10-k and 8-k docs. I’ve pulled out the structured data separately, and now looking at breaking up the text into paragraphs to get embeddings. Why are you using inverted index style text search instead of embeddings? I can see doing both perhaps…
- nnechm 3y ago
  Embeddings did not really work well enough for me, let us say you are looking for numerical text, so for e,g get me AMD's revenue from March 2022. The embedding representation needs to understand that March 2022 together is way more important than March alone, I often ended up with March 2021 or March 2018 as being closer because the text might have multiple terms containing 'revenue' or multiple 'march'... Perhaps I could have improved it, but that did not seem like the right path to go down for accuracy.. This was way worse for e.g when I am looking for ECB statements, they can refer to an older date in their current report and it caused all sorts of trouble :). An initial fix was to basically mention March several times so that the search returns that... :)
DoingIsLearning 3y ago
I wonder if this will lead to fillings that use even more obfuscation in fillings with convoluted wording and obscure language whenever companies need to disclose bad news.
- infecto 3y ago
  Highly doubtful but you never know. Filings are already highly scrutinized.
- blitzar 3y ago
  Almost certainly. Filings are written to communicate a specific message - if the Ai pass was to interpret a hidden meaning, filings would get rewritten to indicate the desired message.
epups 3y ago
What about hallucinations though? If the LLM missed just one number, which at this point is very likely, you would get totally distorted information. Sorry if I missed something, but skimming through the site I didn't find any information about this specific issue.
- nnechm 3y ago
  As the answers are from the text, we can avoid hallucinations for the most part. I have not experienced made up numbers, instead errors are numbers that are typically misplaced, where it gives you revenues for the last 6 months instead of the last 3 months, when they are both next to each other in a table and so on... or the same numbers for different quarters and so on. From my experience, GPT4 has been very good in following instructions and doesn't make up numbers which Bard is much more susceptible to. Bard relies heavily on snippets from the web search and completes the rest... I am pretty sure if google wanted to train it, it would get all the answers right, but the way its designed, it gets one or two numbers and makes the rest up rather hilariously :) and even adds a breakdown which is also made up.. You can take a look here : https://imgur.com/a/vDxOV9D https://imgur.com/a/vDxOV9D
alxdrlitreev 3y ago
We've recently made COFIN AI — an AI based on ChatGPT that reads hundreds of pages of SEC filings such as 10Ks and 10Qs and checks investor calls to assist you with investment research and strategy evaluation. Pretty much similar (almost the same) to something you did. Check this out: https://cofinapp.com https://cofinapp.com Let me know what you think!
- i_like_pie1 3y ago
  asked 3 questions 1st question = got the ticker wrong. asked about company A - gave info on B 2nd question = please give info on A. it says sure 3rd question = please give info on A. pops a modal asking to pay
  - nnechm 3y ago
    I am assuming this is on Cofin and not on beatandraise.com, if so, please let me know, I shall fix it. You have 20 chats etc.
    - i_like_pie1 3y ago
      you are right. i made a mistake. its cofin and i commented on wrong post. messed with yours and had a good experience = false alarm
  - andreev_io 3y ago
    Shoot me a message at ilya@andlabs.co.uk. Happy to give you a free subscription. We also just dropped our price to $1.99 per month from the previous $49.99. Enjoy.
- nnechm 3y ago
  Thanks for letting me know, i guess you use a different strategy of querying by company and by document. I see this when I try to get Apple's rest of asia pacific revenues for e.g I guess it really depends on how your targeted user would use it... :) https://imgur.com/a/uEgLUpz https://imgur.com/a/uEgLUpz
manuelabeledo 3y ago
I may be missing something, but looking at the examples in the website, isn't this just a fancy worded search? What kind of data can be extracted from these filings, that is not available using "traditional" methods?
- halflings 3y ago
  Yes, this looks like a (hopefully) better information retrieval system than running CTRL+F over a PDF. Nothing wrong with that :) information retrieval is hard. And CTRL+F won't render a table for queries like "what is the % increase of revenue over the last 3 quarter"-type queries.
  - nnechm 3y ago
    Yes, information retrieval is hard :) A lot of people ask for 'can you get Apple's revenues by product category for all of 2020', how do you get the smallest piece of text that has the most information about product category and revenues ?I can get a lot of text, but that would mean I probably run over the context limit and so on :)
- nnechm 3y ago
  That's a good question, for e.g if you want to get Microsoft's revenues by product category or apple's revenues in greater China. https://imgur.com/a/LdQkt7j https://imgur.com/a/LdQkt7j You can also do this across time with constraints on context limits ... What is a traditional method, would you search the string or would you find the named entity (using NLP) and look for the entity ? or do you mean a ctrl +F in the document ? I guess the premise of LLMs, is that we have an intelligence that can read and write, so it can do this in an automated way in different applications. In this case, I am trying to automate and speed up the investment research process, along the way, we can create our own dataset of financial data that can be generated on the fly as necessary. Also, how would you extract an income statement from the text, also in the same img using a traditional method ? I personally find it magical it can do that, it knows where it ends and so on...
gen220 3y ago
Beware Gell-Mann amnesia when assessing how useful LLMs are for investing research. Don't forget the instinctual response you have to LLMs' supposed abilities to write software projects, when assessing LLMs' abilities to comprehend and extract the interesting/relevant parts of a 10-Q/K. IME, the most useful software tools in investing are analogous to the most useful software tools in software-building: the IDEs, libraries and frameworks that make an increasingly-expert human more effective. Unfortunately, there aren't any long-term-valuable shortcuts to attaining expertise in any domain, but there are an increasing number dead ends that really look like shortcuts.
- nnechm 3y ago
  I think you are referring to the more hyped version where somehow LLMs can figure out how to get the most relevant information from 10Ks and do most of the investment process. My mental model is simply that we have an assistant, an amazing assistant that can read pretty well, and we can use them in the investment process.
jatinshah 3y ago
The main use case highlighted here seems to be retrieving quantitative data from financial reports and then doing sophisticated analysis (trendlines, forecasting etc) using ChatGPT (basically what’s currently done in Excel). I doubt any finance professional would want to move from Excel to ChatGPT for these use cases. A secondary issue is that numbers in financial reports need to be standardized before they can be used for analysis - say comparing ROE of a stock with industry average. That’s not possible with numbers from raw reports.
- nnechm 3y ago
  I don't think excel can be defeated :) I certainly do not expect financial professionals to switch to a chat interface. When you login into capitaliq or factset or look at a bloomberg screen, you access data, you can then do the same here. Excel plugins go on top which can then get this data into excel to build models. The api that powers this app, can also send data into excel for instance. Data can be copied already directly from the chat box into excel, maintaining the table format for e.g Regarding standardization, I think data was standardized not to enable comparison but simply to fit into the same schema for every industry. Most companies within the same industry report the same way. REITs do not report like SaaS, but the existing datasets put it all into one set. Raw data from source is always better as you can convert it to standardized but you can't go back...
  - jatinshah 3y ago
    If this product is not meant for finance professionals then who is the target customer? Retrieving numbers via ChatGPT and then feeding it into Excel via APIs seems like a very odd thing to do. Especially when most of these numbers are already available on Yahoo finance for individual investors and CapitalIQ for professionals.
    - nnechm 3y ago
      :) I guess its not very odd to say investment research is a lot of reading and LLMs are already very good at reading. So there is little doubt that LLMs will change the investment research process. Have you used CapitalIQ, do you use it on the browser or in excel. If you use it on the browser, the experience using beatandraise.com is already better for some cases. For instance, try getting Apple's Rest of Asia Pacific revenues which apple has been highlighting is their main growth region this year, you can pull that out easily. https://imgur.com/a/rRnrB4u https://imgur.com/a/rRnrB4u Can you do that on capitaliq, you would have to hope their standardized tables include this...
youniverse 3y ago
I know of this thing doing the same, been out a month or two. Haven't used it personally. https://aitickerchat.com/# https://aitickerchat.com/#
hbcondo714 3y ago
Great to see this and the other SEC Filing LLMs in the comments here. I run a freemium SEC Filings website at https://Last10K.com https://Last10K.com to help investors read 10K/Qs more efficiently. Earlier this year, we partnered with https://edgar-gpt.ai https://edgar-gpt.ai for summarizing 10K/Qs. The summaries[1] when clicked on, would then scroll to and display the actual verbiage in the SEC Filing. This feature stayed on the site for a couple months but ultimately we mutually decided to remove it since the traction didn't meet the costs. I'm open to revisiting this topic all in the name of making verbose financial disclosures easier to understand for all of us. [1] https://last10k.com/chatgpt.png https://last10k.com/chatgpt.png

118 points by nnechm 3y ago

I have been working on getting ChatGPT to answer questions that equity research analysts, investors would like to get from SEC filings. The application uses a combination of hybrid text search and LLMs for completion and does not rely much on embedding based distance searches. A core assumption underlying this is that LLMs are already pretty good and will continue to get better at reading texts. If provided with the right thing to read, they will do very well on 'reading comprehension'. Open ended writing is more susceptible to errors, especially in questions related to finance. For e.g google's revenues are just as likely to be 280.2 billion vs 279 billion in a probabilistic model that guesses the next part of the sentence - Google's revenues for FY 2022 are .... So this leaves us with the main problem to solve; Serving the right texts to the LLM aka text search. Once the right text is served, we can generate any pretty much anything in the text, Income statements, ceo comments, accounts payable on the fly, For e.g try - `can you get me Nvidia and AMD's income statement from March 2020 ?` as in here. https://imgur.com/gallery/H8Vfd5X https://imgur.com/gallery/H8Vfd5X A few more examples, Apple's sales in China, Google's revenue by quarter: https://imgur.com/a/oCCay3o https://imgur.com/a/oCCay3o Currently, the application supports ~8k companies that are registered with the SEC. Pdfs are still work in progress, so tesla etc don't work as well. The stack is Nextjs on Supabase. So Postgres's inbuilt text search does a lot of heavy lifting. If one thinks of the bigger picture, we can extend/improve this to pdfs and the entire universe of stocks and more. a.k.a a big component of what CapitalIQ, Factset, Bloomberg and Reuters do can now be generated on the fly accurately for a fraction of the cost. Generating graphs with gross margin increasing etc are just one step further and stuff like EV/Ebitda, yet another step further, as one can call a stock pricing api for each date of the report. I would guess a number of LLM applications follow a similar process, ask a question --> LLM converts to query --> datalakes/bases --> searching and serving texts --> answer. Goes without saying, I would appreciate any feedback, especially from those who are building stuff that looks architecturally similar :) !

paulkon 3y ago
Could you share more about the prompt you use to get GPT-4 to return a query from the user's question? This is fed into postgres full text search?
- nnechm 3y ago
  I am using openai's latest functions api, so you can get it to return arguments that will ensure that you get a json, it works pretty well most of the time. The json would then be used to fetch a report from a database.
anotherhue 3y ago
I've been trying something similar with parliamentary debates. They're long winded, often full of empty speech, and a chore to read. The LLMs are able to hone in on the details and provide interesting responses like "What questions were asked of the minister that they failed to address" and "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer" Crucially, they're well transcribed: https://api.oireachtas.ie/ https://api.oireachtas.ie/
- nnechm 3y ago
  I think one thing you can try is to figure out what lies where, so chunking arbitrarily will not work as well as chunking with headings for e.g For e.g, if the question is: "What should the opposition leader have mentioned in their response that the minister would have found difficult to answer." An embedding based search will find it fairly difficult to match this against a text. Based on my experience, you have to figure out what is a nonanswer first (i don't think that's easy but gpt4 is very good at a lot of language based stuff.) you can try Q1: question A1: Answer then prompt GPT, do you think A1 answers the question and then save it. And then, Q1: question, A1: Answer, Q2: Follow up based on the following questions, do you think A1 answers Q1 and then save it to a db. You can then augment it in the code with our own knowledge of how politicians lie, using certain words etc :) to improve what gpt4 might miss...

halotrope 3y ago

This looks great! We have a similar tool on https://markets.sh https://markets.sh but more geared towards news and general information about stocks (You can ask e.g "What are the current issues with the Disney company?") We found the hardest part about this is the pre-ranking of sections and finding the relevant information. Do you feed the whole report into OpenAI?

nnechm 3y ago
Thanks, I don't feed the whole report in, unless necessary. Context size is a real limiting factor :), so you need to split it up into chunks that are meaningful on their own and find those relevant to the question. In this case, what are the current issues with the Disney company, This is how I would do it :) I get all the news articles related to Disney, this would then have sections (depends if you are using a json or a scraper), now you would then need to pick only those chunks and the surrounding ones (this would be an on the fly search) and then pass it to openai... This one is a very open ended question, this is what I get in beatandraise.com :) https://imgur.com/a/dqmcrRA https://imgur.com/a/dqmcrRA
- nnechm 3y ago
  I don't do handle news articles, and was just focusing on filings, so my answer would look for a filing. That's why it looks the way it does :)

refurb 3y ago

I'm pretty sure a primitive version of this has been done for a long time by hedge funds. You basically monitor the SEC website for when filings are made public, quickly parse the document in some automated way and make trades based on the output. It wouldn't surprise me if some hedge fund has been trying to build something with LLMs for past few years (and possibly already deployed it).

nnechm 3y ago
You are probably right, but there has been nothing like gpt4 before, having seen some versions before I would wager that their entire application can probably be redone for a fraction of the cost and it would work better... :)
xcxcx 3y ago
This was done by hand by people like Footnoted. They received SEC filings using a key red flag word filter into a shared Gmail account with special attention for filings done on Friday night or ahead of the holidays. I think using LLMs to ask a company about their company figures is missing the point. After all, XBRL already exists, and that data is widely accessible anyways. Where LLMs would add more value is analyzing changes in different sections of a company's filings, such as Risk Factors, MD&A, or governance filings.
- nnechm 3y ago
  Yes, I think the broader point is you have these readers available on call via api, so you can do all that you suggested and more...
infecto 3y ago
These already exist and are not primitive. Groups have been applying NLP and analysis against text for longer than you would think.
- nnechm 3y ago
  Yes, they are definitely not primitive, but considering how the breakthrough in LLMs happened, i.e MSFT, GOOGL were working on it, even as late as 2021, Google's BERT was not really there (at least the one they showed the public in the blog from Google research). So the sudden jump from ok to great thanks to openAI would probably mean a lot of existing systems also need to do that.

sroussey 3y ago

I am working on something similar. I have all the 10-k and 8-k docs. I’ve pulled out the structured data separately, and now looking at breaking up the text into paragraphs to get embeddings. Why are you using inverted index style text search instead of embeddings? I can see doing both perhaps…

nnechm 3y ago
Embeddings did not really work well enough for me, let us say you are looking for numerical text, so for e,g get me AMD's revenue from March 2022. The embedding representation needs to understand that March 2022 together is way more important than March alone, I often ended up with March 2021 or March 2018 as being closer because the text might have multiple terms containing 'revenue' or multiple 'march'... Perhaps I could have improved it, but that did not seem like the right path to go down for accuracy.. This was way worse for e.g when I am looking for ECB statements, they can refer to an older date in their current report and it caused all sorts of trouble :). An initial fix was to basically mention March several times so that the search returns that... :)

DoingIsLearning 3y ago

I wonder if this will lead to fillings that use even more obfuscation in fillings with convoluted wording and obscure language whenever companies need to disclose bad news.

infecto 3y ago
Highly doubtful but you never know. Filings are already highly scrutinized.
blitzar 3y ago
Almost certainly. Filings are written to communicate a specific message - if the Ai pass was to interpret a hidden meaning, filings would get rewritten to indicate the desired message.

epups 3y ago

What about hallucinations though? If the LLM missed just one number, which at this point is very likely, you would get totally distorted information. Sorry if I missed something, but skimming through the site I didn't find any information about this specific issue.

nnechm 3y ago
As the answers are from the text, we can avoid hallucinations for the most part. I have not experienced made up numbers, instead errors are numbers that are typically misplaced, where it gives you revenues for the last 6 months instead of the last 3 months, when they are both next to each other in a table and so on... or the same numbers for different quarters and so on. From my experience, GPT4 has been very good in following instructions and doesn't make up numbers which Bard is much more susceptible to. Bard relies heavily on snippets from the web search and completes the rest... I am pretty sure if google wanted to train it, it would get all the answers right, but the way its designed, it gets one or two numbers and makes the rest up rather hilariously :) and even adds a breakdown which is also made up.. You can take a look here : https://imgur.com/a/vDxOV9D https://imgur.com/a/vDxOV9D

alxdrlitreev 3y ago

We've recently made COFIN AI — an AI based on ChatGPT that reads hundreds of pages of SEC filings such as 10Ks and 10Qs and checks investor calls to assist you with investment research and strategy evaluation. Pretty much similar (almost the same) to something you did. Check this out: https://cofinapp.com https://cofinapp.com Let me know what you think!

i_like_pie1 3y ago
asked 3 questions 1st question = got the ticker wrong. asked about company A - gave info on B 2nd question = please give info on A. it says sure 3rd question = please give info on A. pops a modal asking to pay
- nnechm 3y ago
  I am assuming this is on Cofin and not on beatandraise.com, if so, please let me know, I shall fix it. You have 20 chats etc.
  - i_like_pie1 3y ago
    you are right. i made a mistake. its cofin and i commented on wrong post. messed with yours and had a good experience = false alarm
- andreev_io 3y ago
  Shoot me a message at ilya@andlabs.co.uk. Happy to give you a free subscription. We also just dropped our price to $1.99 per month from the previous $49.99. Enjoy.
nnechm 3y ago
Thanks for letting me know, i guess you use a different strategy of querying by company and by document. I see this when I try to get Apple's rest of asia pacific revenues for e.g I guess it really depends on how your targeted user would use it... :) https://imgur.com/a/uEgLUpz https://imgur.com/a/uEgLUpz

manuelabeledo 3y ago

I may be missing something, but looking at the examples in the website, isn't this just a fancy worded search? What kind of data can be extracted from these filings, that is not available using "traditional" methods?

halflings 3y ago
Yes, this looks like a (hopefully) better information retrieval system than running CTRL+F over a PDF. Nothing wrong with that :) information retrieval is hard. And CTRL+F won't render a table for queries like "what is the % increase of revenue over the last 3 quarter"-type queries.
- nnechm 3y ago
  Yes, information retrieval is hard :) A lot of people ask for 'can you get Apple's revenues by product category for all of 2020', how do you get the smallest piece of text that has the most information about product category and revenues ?I can get a lot of text, but that would mean I probably run over the context limit and so on :)
nnechm 3y ago
That's a good question, for e.g if you want to get Microsoft's revenues by product category or apple's revenues in greater China. https://imgur.com/a/LdQkt7j https://imgur.com/a/LdQkt7j You can also do this across time with constraints on context limits ... What is a traditional method, would you search the string or would you find the named entity (using NLP) and look for the entity ? or do you mean a ctrl +F in the document ? I guess the premise of LLMs, is that we have an intelligence that can read and write, so it can do this in an automated way in different applications. In this case, I am trying to automate and speed up the investment research process, along the way, we can create our own dataset of financial data that can be generated on the fly as necessary. Also, how would you extract an income statement from the text, also in the same img using a traditional method ? I personally find it magical it can do that, it knows where it ends and so on...

gen220 3y ago

Beware Gell-Mann amnesia when assessing how useful LLMs are for investing research. Don't forget the instinctual response you have to LLMs' supposed abilities to write software projects, when assessing LLMs' abilities to comprehend and extract the interesting/relevant parts of a 10-Q/K. IME, the most useful software tools in investing are analogous to the most useful software tools in software-building: the IDEs, libraries and frameworks that make an increasingly-expert human more effective. Unfortunately, there aren't any long-term-valuable shortcuts to attaining expertise in any domain, but there are an increasing number dead ends that really look like shortcuts.

nnechm 3y ago
I think you are referring to the more hyped version where somehow LLMs can figure out how to get the most relevant information from 10Ks and do most of the investment process. My mental model is simply that we have an assistant, an amazing assistant that can read pretty well, and we can use them in the investment process.

jatinshah 3y ago

The main use case highlighted here seems to be retrieving quantitative data from financial reports and then doing sophisticated analysis (trendlines, forecasting etc) using ChatGPT (basically what’s currently done in Excel). I doubt any finance professional would want to move from Excel to ChatGPT for these use cases. A secondary issue is that numbers in financial reports need to be standardized before they can be used for analysis - say comparing ROE of a stock with industry average. That’s not possible with numbers from raw reports.

nnechm 3y ago
I don't think excel can be defeated :) I certainly do not expect financial professionals to switch to a chat interface. When you login into capitaliq or factset or look at a bloomberg screen, you access data, you can then do the same here. Excel plugins go on top which can then get this data into excel to build models. The api that powers this app, can also send data into excel for instance. Data can be copied already directly from the chat box into excel, maintaining the table format for e.g Regarding standardization, I think data was standardized not to enable comparison but simply to fit into the same schema for every industry. Most companies within the same industry report the same way. REITs do not report like SaaS, but the existing datasets put it all into one set. Raw data from source is always better as you can convert it to standardized but you can't go back...
- jatinshah 3y ago
  If this product is not meant for finance professionals then who is the target customer? Retrieving numbers via ChatGPT and then feeding it into Excel via APIs seems like a very odd thing to do. Especially when most of these numbers are already available on Yahoo finance for individual investors and CapitalIQ for professionals.
  - nnechm 3y ago
    :) I guess its not very odd to say investment research is a lot of reading and LLMs are already very good at reading. So there is little doubt that LLMs will change the investment research process. Have you used CapitalIQ, do you use it on the browser or in excel. If you use it on the browser, the experience using beatandraise.com is already better for some cases. For instance, try getting Apple's Rest of Asia Pacific revenues which apple has been highlighting is their main growth region this year, you can pull that out easily. https://imgur.com/a/rRnrB4u https://imgur.com/a/rRnrB4u Can you do that on capitaliq, you would have to hope their standardized tables include this...

youniverse 3y ago

I know of this thing doing the same, been out a month or two. Haven't used it personally. https://aitickerchat.com/# https://aitickerchat.com/#

hbcondo714 3y ago

Great to see this and the other SEC Filing LLMs in the comments here. I run a freemium SEC Filings website at https://Last10K.com https://Last10K.com to help investors read 10K/Qs more efficiently. Earlier this year, we partnered with https://edgar-gpt.ai https://edgar-gpt.ai for summarizing 10K/Qs. The summaries[1] when clicked on, would then scroll to and display the actual verbiage in the SEC Filing. This feature stayed on the site for a couple months but ultimately we mutually decided to remove it since the traction didn't meet the costs. I'm open to revisiting this topic all in the name of making verbose financial disclosures easier to understand for all of us. [1] https://last10k.com/chatgpt.png https://last10k.com/chatgpt.png