10 ms·
LlamaCloud and LlamaParse
- coding123 3y agoWhat's a RAG application
- seldo 3y agoRetrieval-Augmented Generation, where you ask an LLM to answer a question by giving it some context information that you have retrieved from your own data rather than just the data it was trained on.
- simonw 3y agoRAG stands for Retrieval Augmented Generation. It's the trick where a user asks you a question: "Who worked on the billing UI refresh last year?" - and you turn that question into a search against a bunch of private documents, find the top matches, copy them into a big prompt to an LLM and ask it to use that data to answer the user's question. There's a HUGE amount of depth to building this well - it's one of the most actively explored parts of LLM/generative-AI at the moment, because being able to ask human-language questions of large private datasets is incredibly useful.
- pierre 3y agoI'm part of the team that build LlamaParse. It's net improvement compare to other PDF->Structured Text extractors (I build several in the past, includig https://github.com/axa-group/Parsr https://github.com/axa-group/Parsr). For character extraction, LlamaParse use a mixture of OCR / character extraction from the PDF (it's the only parser I'm aware of that address some of the buggy PDF font issues, check the 'text' mode to see raw document before reconstruction), use a mixture of heuristic and Machine learning models to reconstruct the document. Once plug with a Recursive retrieval strategy, allow you to get Sota result on question answering over complexe text (see notebook: https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb https://github.com/run-llama/llama_parse/blob/main/examples/...). AMA
- binarymax 3y agoCool! Which OCR engine/model do you use?
- pierre 3y agoEasyOCR, may switch to paddleOCR in the future.
- vikp 3y agoYou may want to try https://github.com/VikParuchuri/surya https://github.com/VikParuchuri/surya (I'm the author). I've only benchmarked against tesseract, but it outperforms it by a lot (benchmarks in repo). Happy to discuss. You could also try https://github.com/VikParuchuri/marker https://github.com/VikParuchuri/marker for general PDF parsing (I'm also the author) - it seems like you're more focused on tables.
- raffraffraff 3y agoHow does surya compare to AWS Textract? A previous employer went through a bunch of different OCRs and ended up using Textract because they found it to be the most accurate overall.
- pryelluw 3y agoThanks for sharing.
- 3y ago
- bx376 3y agoWhat will the pricing be like?
- ldjkfkdsjnv 3y agoModern playbook: 1. Build janky open source code base 2. Sell compute to run it 3. Build features that create compute lock in (vercel is a master at this)
- tslmy 3y agoHere's an alternative: Spend seed round investments on building a solid software but not building an income stream that can satisfy investors, thus not receiving any new funding and let the company die.
- gbickford 3y agoThere's gotta be somewhere in the middle. Vercel's movements feel a lot like the "Embrace, extend, and extinguish" playbook. Maybe there is a class of developer out there that doesn't get spooked by that but it definitely has created an adversarial place for Vercel in my mind. I feel like I need to be careful when touching anything Vercel have touched so that I don't fall into a trap.
- verdverm 3y agoI just stopped using their NextJS project because you can no longer self host the middleware, they now only support edge runtime and several libraries don't work with it. I'm calling this situation Fauxpen Source. The recent moves definitely feel anticompetitive or at least trying to force you into using their products I'm migrating to vite+vike (next/nuxt like experience for any framework)
- Rauchg 3y agoMiddleware does work with self-hosting[1]. It’s a more limited runtime that’s based on web standard APIs, which creates optionality for running it in high performance / resource constrained scenarios. [1] https://nextjs.org/docs/app/building-your-application/deploying#middleware https://nextjs.org/docs/app/building-your-application/deploy...
- johnsutor 3y agoI wonder how LlamaParse compares head to head with https://unstructured.io https://unstructured.io
- justanotheratom 3y agonot clear to me why this got downvoted. sensible question.
- infecto 3y agoI would also like how it compares to any of the commercial offerings from Azure/AWS/GCP. They all have document parsing tools that I have found better than tools like unstructured. Sure you don't have some of the "magic" of segmenting text for vectorization and RAG but imo thats the easy part. The hard part is pulling data, forms, tables, text out of the PDF which I find the cloud tools to do a superior job.
- bugglebeetle 3y agoNothing beats Google text extraction tasks in my testing, especially for East Asian languages. I wish something else worked better, because their services are fairly expensive.
- infecto 3y agoI mostly have used Textract but have found all 3 to be fairly similar in accuracy with differences in structure and compatible languages. I think my call out here is that I don't think any of these libraries, llamaindex or unstructured, can compete in this area. I would rather use GCP/Azure/AWS to define the structure from a PDF and these for the rag portion if anything.
- youngNed 3y agoBe careful with unstructured: https://github.com/Unstructured-IO/unstructured/blob/d11c70cf83fdb8a08fed2cf01c6c0bd114d817df/unstructured/utils.py#L287-L319 https://github.com/Unstructured-IO/unstructured/blob/d11c70c... from: https://github.com/open-webui/open-webui/issues/687 https://github.com/open-webui/open-webui/issues/687
- srameshc 3y agoI don't understand why post this on medium ? Medium doesn't let me even read anymore. If you have a blog post on it so your audience can reach you.
- eferreira_ 3y agoAlso, have a X (formerly Twitter) thread: https://x.com/llama_index/status/1759987390435996120?s=20 https://x.com/llama_index/status/1759987390435996120?s=20
- diggan 3y agoWhich also isn't really available to unregistered users, can only see the first tweet: https://i.imgur.com/SJA2Gzs.png https://i.imgur.com/SJA2Gzs.png
- seldo 3y agoWe are planning to move our blog off of Medium (we've been busy!), but this post is public so you can actually just click through the nag screen if you see one.
- deleted 3y ago[deleted]
- Alifatisk 3y agoRight! Both Medium and Twitter is hostile you don’t join them. Is it really that much of a burden to host a mini blog?
- diggan 3y ago> Is it really that much of a burden to host a mini blog? The problem is not that it's a hassle to setup and maintain, but that discovery and network effects are a lot smaller when self-hosting. Being on Twitter and Medium gives you a lot more readers from the get-go, just because of the built-in discovery. I still self-host my own stuff, but I would lie if I said I didn't understand why people use Twitter and Medium for hosting their content.
- miohtama 3y ago> PDFs are specifically a problem: I have complex docs with lots of messy formatting. How do I represent this in the right way so the LLM can understand it? 40 years after PostScript and this is still a problem that one needs to throw AI at. I feel the software development and human-computer interaction took a wrong turn along the way. What happened to the semantic web?
- avhon1 3y agoIt turns out that it takes thought effort to semantically tag/classify everything consistently and completely, so rather than make the decisions, it's easier to just not do it.
- madeofpalk 3y agoWhat? We still have 'the web'. PDFs are something different and separate.
- lxe 3y agoLlamaParse solves exactly the problem I've encountered over and over with RAG. Getting structured info from unstructured data is a pain.
- pknerd 3y agoSorry for offtopic: Are there any LLM services that I can use in cloud similar to OpenAI? I do not have good enough Macbook to run different models locally
- tslmy 3y agoHmmm... OpenAI itself? Did you intend to rule out OpenAI from consideration? You mentioned hardware being a constraint, but that doesn't tell me why you specifically wanted to find an alternative to OpenAI.
- simion314 3y agoNot OP, in my case OpenAI does not want my money, they only accept credit cards. For example netflix wants my money so they have more choices. Also I would like to pay for an equivalent alternative that is less censored, like ChatGPT had a bug one day that it refused to tell me how to force a type cast in TypeScript, it showed me a moderation error. So I want an AI that is targeted for adults and not children in some religious school in USA.
- 123yawaworht456 3y agothere is a somewhat unfiltered GPT4 at Azure, but they really don't want anybody's money (afaik only "trusted" corporate entities can access it) at this time, your only option is local models. if you don't have the hardware to run them yourself, there are plenty of hosts - poe/perplexity/together etc. llama3 is (hopefully) coming soon, and if it has improved as much as llama2 improved over llama1, and provides at least 16k baseline context size, it will be in between gpt3.5 and gpt4 in terms of quality, which is mostly enough.
- simion314 3y agoYes, there are 2 issues for me, my hardware is not powerful enough, only 8Gb vram and the models are still not intelligent enough. At this moment I have opened tabs for different websites and when I ahve a question I compare them and see the state. I would like a model that would say I do not know more often then respond with the wrong thing, also I would like it to follow instructions , now I ask them to "rewrite previous response but without X" and they respond "sure, here is the response without X " and they do not followed the instruction, like they are "hard coded" to do X. An example for X is "do not add a summary or conclusion"
- technics256 3y agoLlamaParse looks nice. Is there way to return page numbers also with the markdown? This is important for our use case.
- behnamoh 3y ago> This is where LlamaParse comes in. We’ve developed a proprietary parsing service that is incredibly good at parsing PDFs with complex tables into a well-structured markdown format. This is my problem with projects that start off as open source and become famous because of their community contributions and attention, then the project leaders get that sweet VC money (or not) and make something proprietary. We've seen it with Langchain and several other "fake open source" projects.
- siquick 3y agoLlamaParse is proprietary but the main LI package isn’t and you don’t need the former to use the latter. Why shouldn’t they make money? LI is a fantastic way to do RAG.
- zmmmmm 3y agoI don't disagree but I think it's not a question of "why" but "how". It could still be licensed in a restricted way, but keeping secret how it works is unfortunate - it breaks the chain of learning that is happening across the open ecosystem and, if the technique is any good, all it does is force open models to build an actually open equivalent so that further progress can be made (and if it's not really any good then it's snake oil, which is worse). Even if it's great it essentially becomes a dead end for the people who actually need and want an open model ecosystem.
- baby 3y agowhat is RAG?
- doublerabbit 3y agoretrieval augmented generation. Explained by gpt itself as if you were a teddy bear. ---- Okay little teddybears, let me explain what retrieval augmented generation is in a way you can understand! You see, sometimes when big AI models like Claude want to talk about something, they may not know all the facts. But they have a friend named the knowledge base who knows lots of information! When Claude wants to talk about something new, he first asks the knowledge base "What do you know about X?". The knowledge base looks through all its facts and finds the most helpful ones. Then it shares them with Claude so he has more context before talking. This process of Claude asking the knowledge base for facts is called retrieval augmented generation. It helps Claude sound smarter and avoid mistakes, because he has extra information from his knowledgeable friend the knowledge base. The next time Claude wants to chat with you teddybears, he will be even better prepared with facts from the knowledge base to have an interesting conversation!
- baby 3y agois this related to vector dbs?
- _pdp_ 3y agoQuestion, why build this when you can use LLMS to extract the data in the most appropriate format to begin with? Isn't this a bit redundant? Perhaps it makes sense in the short term due to cost but in the long run this problem can be solved generically with LLMS.
- lolpanda 3y agoI think LlamaParse is trying to solve a hard problem. Many enterprise customers I know have strong need to parse PDF files and extract data accurately. I found the interface a bit confusing. From your blog post, LlamaParse can extract numbers in tables, but it appears that the output isn't provided in tabular format. Instead, access to these numbers is only available through a question-answering. Is this accurate?
- cheesyFishes 3y agoThe output is either text or markdown, and from there you can handle it however you need. In LlamaIndex for example, there are a a few markdown-specific classes that work well with this. You can find an example over in the repo -- https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb https://github.com/run-llama/llama_parse/blob/main/examples/...
- lolpanda 3y agoI was hoping to get structured data. For example, parsing an voice will give results like {"title"... "line_items": [...], "date":...}
- kurts_mustache 3y agoHmm, I have to say I'm pretty unimpressed with my initial experience here. 1. The sign up with email just endlessly redirected, click link in email, ask to sign up with email, put in email, click link in email, etc. 2. Fine, I'll sign in with Google. 3. A PDF parser? Seriously that's what all this fuss is about? There are so many options already out there, PDFBox, iText, Unstructured, PyPDF, PDF.js, PdfMiner not to mention extraction services available from the hyperscalers. Super confused why anyone needs this.
- verdverm 3y agoLLaMA Index is way more than a PDF parser. It's the most widely used RAG tool chain and their cloud looks to be a managed version of that. Specific to the parser, they do show where tools like those you mentioned fail and their LLM based parser captures the full data the aforementioned miss.
- kurts_mustache 3y agoYeah, but their platform is basically a janky PDF parser which is why I don't understand what the hype is about. It's easy to cherry pick a PDF for marketing purposes and claim you're better. I didn't miss it, I just don't believe marketing announcements at face value. I tried their parser on a PDF with a bit of complex formatting like multiple columns, tables and a couple images and it choked, spitting out one big markdown header with jumbled text. Not impressed.
- cheesyFishes 3y agoThere is a PDF parser, LlamaParse, (which is open to everyone), and a managed ingestion/retrieval service, that is currently invite-only. Planning broader releases in the future for sure.
- asukla 3y agoTo get good RAG performance you will need a good chunking strategy. Simply getting all the text is not good enough and knowing the boundaries of table, list, paragraph, section etc. is helpful. Great work by llamaindex team. Also feel free to try https://github.com/nlmatics/llmsherpa https://github.com/nlmatics/llmsherpa which takes into account some of the things I mentioned.
- Oras 3y agoIsn't that similar to what AWS Textract does? They have this functionality of parsing and querying info from tables and forms. I'm sure that for LI, having it as part of a workflow and history to retrieve for RAG makes it easier for users, but why reinvent the wheel? Unless I missed something