4 ms·
Show HN: Embedding model for PDF page retrieval
We have fine-tuned an embedding model that can help to search for relevant pages in a document better than standard embedding models. It's still a work in progress, so looking for feedback and ideas on how to improve it!
- navaed01 2y agoWhat is the use case for this? Is the value here that you do not need to worry about accurately extracting text from a pdf? Or that you have full knowledge of where text is store as it relates to the original document? Or something different?
- Boriscii 2y agoThere is a growing trend of applying Multimodal LLMs directly to PDF pages without doing extraction first (i.e. classic RAG). We've had this request from multiple customers whose products are supposed to handle visually rich documents like "GPT for pitch decks" or "GPT for investment bankers". The problem is that you must first find the relevant page to feed into the LLM. This helps to avoid the overhead of running an expensive model on each single page. We first attempted to extract text from pages and do a semantic/hybrid search. Still, it felt massively suboptimal since much positional information was lost, so we decided to train our model that could help with this task!
- Boriscii 2y agoAlso, please feel free to take a look at the blog post if you are interested in more technical details! https://blog.bynedocs.com/layoutlm-byne-v0.1-the-new-sota-in-page-retrieval-from-visually-rich-documents https://blog.bynedocs.com/layoutlm-byne-v0.1-the-new-sota-in...
- navaed01 2y agoThanks for the explanation- I have a need for this!