3 ms·

I worked on such a project 8 years ago. I actually ended up building my own OCR engine, after annotating manually about 50 receipts (about 8000 characters if I

by rolisz 6y ago

I worked on such a project 8 years ago. I actually ended up building my own OCR engine, after annotating manually about 50 receipts (about 8000 characters if I remember correctly). Some of the problems I encountered back then is that snapping a picture of a receipt with your phone will result in weird lighting conditions and angles which will mess with the OCR engine. The second problem is that it's hard to keep the receipt straight while taking the picture, so it will be hard to identify lines in the picture, because they will be curved. To some extent, all this is solved by some modern APIs, such as what GCP or AWS offer, for doing OCR for you. But as far as I know, there is still one more challenge: interpreting the text. Inferring what each line is, what's the price for which item (some receipts have the price on the same line, some on the next line, some above) is quite hard. I tried to do it with rules (regexes and lots of ifs), but even a 95% accuracy of the OCR engine will trip you up. You can probably frame this as an ML problem as well, but I don't think you'll find any datasets for this.

wcarss 6y ago
As I mentioned in another comment, some friends of mine worked at a startup that's entirely built around receipt scanning and itemization, and your comment aligns with what I heard from them ad nauseam: receipts are hard, in large part because there's just no standard way of putting things onto them. How do you show subtotals, taxes, and totals? How do you flag that an item is taxable or not? What's part of the header? What format are the numbers? What kind of subtle background text is on the receipt? Is the receipt at an angle? Is the picture taken of just a receipt, or a receipt held up in the air, with stuff behind it? Sometimes, there are lines on receipts that are just meant to be ignored, maybe for old tax regulations that don't exist anymore but were important when the receipt-printing software was written. It's a mess.
rolisz 6y ago
One thing that could help a lot is trying to get the receipt data from some loyalty card some shops have. In Romania, almost all big stores have mobile apps which allow you to export your purchase lists. Granted, some of them have dumb outputs (Lidl give you an image of your receipt, so you still have to do OCR on it, Carrefour gives you a PDF), but it does make the problem much easier. Of course, this won't work for your random corner shop.

by rolisz 6y ago

wcarss 6y ago

As I mentioned in another comment, some friends of mine worked at a startup that's entirely built around receipt scanning and itemization, and your comment aligns with what I heard from them ad nauseam: receipts are hard, in large part because there's just no standard way of putting things onto them. How do you show subtotals, taxes, and totals? How do you flag that an item is taxable or not? What's part of the header? What format are the numbers? What kind of subtle background text is on the receipt? Is the receipt at an angle? Is the picture taken of just a receipt, or a receipt held up in the air, with stuff behind it? Sometimes, there are lines on receipts that are just meant to be ignored, maybe for old tax regulations that don't exist anymore but were important when the receipt-printing software was written. It's a mess.

rolisz 6y ago

One thing that could help a lot is trying to get the receipt data from some loyalty card some shops have. In Romania, almost all big stores have mobile apps which allow you to export your purchase lists. Granted, some of them have dumb outputs (Lidl give you an image of your receipt, so you still have to do OCR on it, Carrefour gives you a PDF), but it does make the problem much easier. Of course, this won't work for your random corner shop.