3 ms·
Ah, yes, I've found pre-processing the PDFs to sanitize against things like that has been helpful. That's a whole other process though.
by ajcp 2y ago
Ah, yes, I've found pre-processing the PDFs to sanitize against things like that has been helpful. That's a whole other process though.
- sumedh 2y agoWhat steps does that involve?
- ajcp 2y agoEssentially what you're already doing, with one more step :) Get PDF > convert (read: rebuild) to TIFF > convert to PDF. In my case all documents to be sent to the LLM (PDFs/Images/emails/etc) are already stagged in a file repository as part of a standard storage process. This entails every document being converted into a TIFF (read: rebuilt cleanly) for storage, and then into PDF upon export. This ensures that all docs are correct and don't maintain whatever went into originally creating them. I've found any number of "PDF" documents are not PDF, while others try and enforce some "protection" that makes the LLM not like the DOCS
- sumedh 2y agoInteresting, I will try the TIFF approach for some of the problems Pdfs I have. Thanks