11 ms·
DeepDoctection: Document extraction and analysis using deep learning models
- cgio 3y agoHow serendipitous. I was looking for something like that recently. Admittedly my use case was much simpler, detecting tables of contents on scanned pdfs that usually don’t have them as links to navigate within the document. Will see if this could help. Is anyone using something else for my use case?
- ensemblehq 3y agoNot sure what your budget is but I’ve used AWS for handling PDFs and it’s been pretty good at detecting content via boundary boxes.
- anamexis 3y agoAWS as in Amazon Web Services? And if so, can you be more specific?
- navanchauhan 3y agoHere are some more options: * AWS Textract [0] * Microsoft Azure [1] * Google Cloud Vision [2] I personally use Azure, combined with OCR correction using GPT to convert a scan of my daily journal (Apple Notes creates a PDF that is nothing but a bunch of images) -> Markdown -> Extract tasks and then add them to my Reminders app using CalDav. Azure has one of the best OCR for handwritten text, but for normal document extraction (read: printed text), any service would do a reasonable job. [0] https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/automatically-extract-content-from-pdf-files-using-amazon-textract.html https://docs.aws.amazon.com/prescriptive-guidance/latest/pat... [1] https://learn.microsoft.com/en-us/azure/data-factory/solution-template-extract-data-from-pdf https://learn.microsoft.com/en-us/azure/data-factory/solutio... [2] https://cloud.google.com/vision/docs/pdf https://cloud.google.com/vision/docs/pdf
- swsieber 3y agoCould you explain how you use GPT for ocr correction?
- navanchauhan 3y agohttps://promptbase.com/prompt/ocr-text-fixer https://promptbase.com/prompt/ocr-text-fixer This is the prompt I bought from promptbase. You basically provide GPT with some examples on possible OCR errors, and then you give it the OCRed text and it tries to correct it
- swsieber 3y agoFascinating. Thanks!
- tevlon 3y agoIsn't it depressing, that we live in 2023 and the predominant document format is pdf, which was invented in 1993 and is optimized for printing? I would love to have a new format, which is easily parseable (like JSON) AND printable (like PDF).
- boredemployee 3y agoCreate one for us :D
- zmmmmm 3y agoat least PDF occasionally contains actual text. My organisation systematically scans everything to TIFF images for archival. So now we are embarking on a major project to OCR the TIFFs to get back the text (!).
- UncleEntity 3y agoMy payroll statement is the same, image wrapped in a pdf document. I’m not sure if they’re being intentionally annoying or if someone thought this was actually helpful for the thousands of independent contractors who track their expenses down to the penny?
- Baeocystin 3y agoConsidering the data format hells I've had to deal with over the years, straightforward TIFF scans don't sound so bad, honestly.
- therealmarv 3y agoAsciidoc, HTMLbook, Docbook are the standards O'Reilly media using to create their books (and PDFs). No need to reinvent the wheel there.
- mr_toad 3y agoI thinks it’s depressing that we’re still thinking of content being containerised as if it still had to be bound in a physical volume instead of being addressable items of information, like a computer naturally stores information.
- fnordpiglet 3y agoInteresting I was just thinking a LLM would do a great job correcting OCR mistakes.
- vivegi 3y agoOCR engines may use a HMM (Hidden Markov Model) for OCR correction.
- fnordpiglet 3y agoYeah, but they don’t do a semantic correction. LLM are extraordinarily more powerful than HMM.
- vivegi 3y agoDepends on context though. An LLM is useless (or not as useful) in OCR for forms where we are trying to extract the name "John Smith" from a name field whereas a HMM trained specifically on name fields may be able to do a better job. LLMs may perform better as a post processing step for running text (such as book pages).
- airocker 3y agoI was just looking for OCR. How does this compare with easyOCR?
- avibhu 3y agoFor what its worth, very high quality OCR from Google's Vision offering costs $0.0015 per page, with 1000 free pages per month. In my experience, it has been signficantly superior to any open source solution.
- airocker 3y agoThanks!
- o_____________o 3y agoWhy this over Document AI?
- dchuk 3y agoWhat’s state of the art for using ML to understand html docs automatically when scraping them? I’ve tinkered with this: https://huggingface.co/docs/transformers/model_doc/markuplm https://huggingface.co/docs/transformers/model_doc/markuplm and it seems useful, but ai and ml is changing so fast right now, anyone know what else is going on? An example use case: given a url, figure out content type (article vs product page for example), if product page, automatically extract all product details and specs without manually mapping xpath or css lookup paths
- ShamelessC 3y agoGPT-4 presumably.
- flir 3y agoI keep bumping in to the context window size. I'm trying to figure out a "compression" step that I can use in the general case, but nothing's very satisfying so far. The mozilla/readability library is a good first step though.
- newswasboring 3y agoWhy do you want to compress this data? What's the final use case here?
- jerrygenser 3y agoCompression in order for the input data to fit context, maybe? For example if context is 4092 and input size is 6000, figuring out the appropriate way to run an operation on all 6000 where context over all 6000 might be relevant to the operation.
- newswasboring 3y agoMaybe I'm not getting it, but I see this as an indexing problem. The goal shouldn't be to fit the entire document in the prompt, we should include relevant part of the doc when we query it. Edit: I'm thinking of something like LlamaIndex
- o_____________o 3y agoI would pay for a simple, competent anything-to-markdown API. Something that could convert PDFs to high quality markdown with tables, etc. I'm using Document AI from Google right now and the ergonomics are awful.
- gcr 3y agoABBYY reader is a fine human-in-the-loop solution. It can at least output from PDF to ePub, and you can imagine going to markdown from there.
- j45 3y agoAnything to markdown is broader than part to markdown Since pdfs are created so many different ways, do you have some examples and links of the pdfs that are awful?
- whyleyc 3y agoIf you just need to convert the files have you thought about using Zamzar (https://dev.zamzar.com/ https://dev.zamzar.com/)? We have a file conversion API that supports DOC/DOCX/ODT/PDF/TEX to Markdown conversion in one line of cURL (or you programming language of choice). (Disclaimer: I'm the product lead for the Zamzar API).
- o_____________o 3y agoThanks I'll check it out. What do you do with PDFs that lock text in images, are you using ML/OCR? And as mentioned, tables?
- whyleyc 3y agoCurrently OCR support is limited to PDF > TXT conversion but we're hoping to add support for other output formats at some point. Feel free to shoot me an email at chris [at] zamzar [dot] com if you'd like to chat further.
- bravura 3y agohttps://pd3f.com/ https://pd3f.com/ is a good PDF converter for the use-cases I tested (academic papers).
- exhibitapp 3y agoI've worked extensively in this space. For those looking for just an OCR solution MSFT's offering "read" is by and far the most accurate. Key-value, table and other information extraction is a much harder problem. Anything that can go wrong in production will. Documents with extra pages, rotated, blacked out, fuzzy. There are many steps that go into making document extraction really e2e. The biggest enterprise users are doing thousand+ of pages a minute and also turn document extraction into a scaling distributed systems problem
- nextworddev 3y agoDo they have human workers for those hard to solve cases in the loop?
- exhibitapp 3y agoYes the solution i worked on had an interface for HITL
- idealism 3y agoA few days ago, IBM announced a new OCR system[1]. Have you by chance compared it to Microsoft's offering? I'm currently looking for the best-in-class OCR solution for scanned PDF documents. [1]: https://www.ibm.com/cloud/blog/exploring-ibms-new-optical-character-recognition-technology https://www.ibm.com/cloud/blog/exploring-ibms-new-optical-ch...
- alsodumb 3y agoCall me biased, but I've learned over time that anything that comes out of the Waston team looks good only in PR statements but sucks at production - especially at tasks like OCR. YMMV.
- 9dev 3y agoDo you have any recommendations for OCR of receipts and grocery bills? I’ve dreamt of having a little app to analyse grocery spending and distribute bills among multiple people, but every time I checked, the state of receipt OCR was surprisingly too bad for this…
- themantalope 3y agoI’ve used this repo. It’s ok. For very simple layouts, it probably works fine. For more complex layouts it fails miserably. I’ve also had cases where it didn’t detect half the text on the page (machine generated text).
- morisy 3y agoAnything you find better for complex layouts?
- elpalek 3y agohttps://github.com/microsoft/table-transformer https://github.com/microsoft/table-transformer This looks interesting as well. Haven't tested yet.