3 ms·
Recently I tied OCR with Opus 4.8. (I know, not technically right tool for the job). All I needed to do was extract dates from receipts. It got about 20% of the
by Insanity 4mo ago
Recently I tied OCR with Opus 4.8. (I know, not technically right tool for the job). All I needed to do was extract dates from receipts. It got about 20% of the dates wrong yet rated all as “high confidence”.
Should have probably tried a more OCR specific model
- nik736 4mo agoOpus is very good at OCR. Way better than the small 1-4B VLMs. If Opus failed, most likely those smaller models will fail as well.
- MostlyStable 4mo agoHow long have you been testing this? Have you noted a large improvement? I tested Opus for this quite a while ago (maybe 4.5? Whatever was out about a year ago), and it performed quite poorly on my use case.
- nik736 4mo agoI have put together an internal benchmark on 1000s of business documents with weird tables, structure, etc. that I run on every relevant model release. Opus 4.8 performs very very well. But it is obviously overkill for the task (and expensive at doing so). I just wanted to respond to the OP.
- apawloski 4mo agoI’m curious what your findings are for the best model for your use case
- Insanity 4mo agoI'm assuming that the reason I didn't have good success rate is because it was not scanned documents, but photographs, and lighting conditions weren't always ideal. I think scanned business documents are a happy-case scenario in a way. (obv, you seem to run it against some complex documents, so that's impressive)
- bongoman42 4mo agoWe use Qwen 7B parameter model for this in production. It works quite well.
- bpodgursky 4mo agoI do not believe this story. Opus 4.8 scanned hundreds of PDFs for me recently with the worst handwriting imaginable. 100% successful, other than one record where even I could not figure out what was written.
- 9cb14c1ec0 4mo agoI believe it. Makes me curious what your prompt was that got such a good result out of Opus.
- jokethrowaway 4mo agolikely just opus being dumbed down to prepare for fable
- bpodgursky 4mo agoI ordered it to not be lazy and to show me result 1 at a time rather than doing it in batch.
- Insanity 4mo agoI do not believe this story, because of the message I just posted above. That's not really productive lol, I'm glad it worked for you but these models are non-deterministic and 'YMMV' very much applies everywhere. I had it parse receipts (in fairness, in variable lightning), all taken from iPhone cameras in the past year. And yeah, not a great job, about 20% failed to get the date correct. (Not outrageously wrong, e.g 05/20/2026 becomes 05/23/2026. YMMV, glad it worked for you.
- bpodgursky 4mo agoAre you sure you weren't using Sonnet or a low-effort reasoning mode?
- Insanity 4mo agoYes, lol
- rsynnott 4mo ago> All I needed to do was extract dates from receipts Was this... not basically a solved problem like 30 years ago? I'm pretty sure the shareware OCR tool that came with a black and white scanner I had at one point would do better than 20% wrong.
- staticman2 4mo agoI don't know about Opus but I can tell you with Gemini the subscription product OCR is apparently not done by the model. It used a separate old fashioned OCR tool and gives bad results in my tests. But with Gemini the API the model does do the OCR resulting in much better accuracy.