The challenge is doing this offline/on-device. There are a billion receipt scanner APIs and services in the cloud that are basically "upload image, get back str

The challenge is doing this offline/on-device. There are a billion receipt scanner APIs and services in the cloud that are basically "upload image, get back structured receipt data", but that's not what you want? Some links... - A 2019 machine vision competition about this exact use case: https://rrc.cvc.uab.es/?ch=13 https://rrc.cvc.uab.es/?ch=13. Some of the submissions in the results table will discuss their methods in depth (usually in Python though). - Implementing this without AI/ML, just using Tesseract (open-source OCR, but poor performance compared to commercial apps or ML): https://pyimagesearch.com/2021/10/27/automatically-ocring-receipts-and-scans/ https://pyimagesearch.com/2021/10/27/automatically-ocring-re... - Using Google's ML-kit (on-device Document Scanner SDK): https://teresa-wu.medium.com/googles-ml-kit-text-recognition-with-sample-app-of-receipts-reading-7fe6dc68ada3 https://teresa-wu.medium.com/googles-ml-kit-text-recognition..., but the tabular data is hard to work with. This writeup goes one step further: https://hackernoon.com/using-google-mlkit-text-recognition-to-scan-tabular-data https://hackernoon.com/using-google-mlkit-text-recognition-t... - 2019 HN discussion on the same: https://news.ycombinator.com/item?id=10338199 https://news.ycombinator.com/item?id=10338199 - A cloud OCR vendor (Nanonets) discussing their ML-based pipeline: https://www.linkedin.com/pulse/how-build-ocr-receipt-scanner-from-scratch-using-deep-olivia-smith https://www.linkedin.com/pulse/how-build-ocr-receipt-scanner... ---------- (Edit: Cloud options) Last time I looked into this (for a similar use case) I didn't get very far, and decided that using the cloudy Nanonets receipts API (https://nanonets.com/ocr-api/receipt-ocr https://nanonets.com/ocr-api/receipt-ocr) or the Google Cloud document classifier (https://cloud.google.com/blog/products/application-modernization/use-cloud-ai-tools-to-read-and-classify-your-receipts/ https://cloud.google.com/blog/products/application-moderniza...) is going to work a lot better and easier than anything I could do on my own. But that's probably no better than Textract. There's just so many entrants in this space, so many huge companies working on their own proprietary APIs, that the small open-source libs don't really stand much of a chance in comparison. It's too bad :(