4 ms·

If you can accept that the machine just make up what it doesn't recognize instead of saying "I don't know," then yes it's solved. (I'm not being snarky. It's a

If you can accept that the machine just make up what it doesn't recognize instead of saying "I don't know," then yes it's solved. (I'm not being snarky. It's acceptable in some cases.)

jakewins 1y ago
But this was very much the case with existing OCR software as well? I guess the LLMs will end up making up plausible looking text instead of text riddled with errors, which makes it much harder to catch the mistakes, in fairness
- rkagerer 1y ago
  Good libraries gave results with embedded confidence levels for each unit recognized.
- wahnfrieden 1y ago
  Existing ocr doesn’t skip over entire (legible) paragraphs or hallucinate entire sentences
  - Davidzheng 1y ago
    rarely happens to me using LLMs to transcribe pdfs
  - criddell 1y ago
    I usually run the image(s) through more than one converter then compare the results. They all have problems, but the parts they agree on are usually correct.
  - KoolKat23 1y ago
    This must be some older/smaller model.
red75prime 1y ago
Just checked it with Gemini 2.5 Flash. Instructing it to mark low-confidence words seems to work OK(ish).
wahnfrieden 1y ago
Do any LLM OCRs give bounding boxes anyway? Per character and per block.
- kelvinjps10 1y ago
  Gemini does but it's not as good as Google vision, and the format it's différent Here it's the documentation https://cloud.google.com/vertex-ai/generative-ai/docs/bounding-box-detection https://cloud.google.com/vertex-ai/generative-ai/docs/boundi... Also Simon Willison Made a blog post that might be helpful https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/ https://simonwillison.net/2024/Aug/26/gemini-bounding-box-vi... I hope that this capability improves so I can use only Gemini API.
- dajonker 1y ago
  Try MinerU 2.5 with two-step parsing. It gives good results with bounding boxes per block. Not sure if you can get it to do more detailed such as word or character level.
KoolKat23 1y ago
These days it does just that, it'll say null or whatever if you give it the option. When it does make it up, it tends to be limitation of the image qualify ( max dpi). Blotchy text and specific typeface make 6's look like 8's, even to the non-discerning eye, a human would think it's an 8, zoom in and see it's a 6. Google's image quality on uploads is still streets ahead of openai for instance btw.

If you can accept that the machine just make up what it doesn't recognize instead of saying "I don't know," then yes it's solved. (I'm not being snarky. It's acceptable in some cases.)

jakewins 1y ago

But this was very much the case with existing OCR software as well? I guess the LLMs will end up making up plausible looking text instead of text riddled with errors, which makes it much harder to catch the mistakes, in fairness

rkagerer 1y ago
Good libraries gave results with embedded confidence levels for each unit recognized.
wahnfrieden 1y ago
Existing ocr doesn’t skip over entire (legible) paragraphs or hallucinate entire sentences
- Davidzheng 1y ago
  rarely happens to me using LLMs to transcribe pdfs
- criddell 1y ago
  I usually run the image(s) through more than one converter then compare the results. They all have problems, but the parts they agree on are usually correct.
- KoolKat23 1y ago
  This must be some older/smaller model.

red75prime 1y ago

Just checked it with Gemini 2.5 Flash. Instructing it to mark low-confidence words seems to work OK(ish).

wahnfrieden 1y ago

Do any LLM OCRs give bounding boxes anyway? Per character and per block.

kelvinjps10 1y ago
Gemini does but it's not as good as Google vision, and the format it's différent Here it's the documentation https://cloud.google.com/vertex-ai/generative-ai/docs/bounding-box-detection https://cloud.google.com/vertex-ai/generative-ai/docs/boundi... Also Simon Willison Made a blog post that might be helpful https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/ https://simonwillison.net/2024/Aug/26/gemini-bounding-box-vi... I hope that this capability improves so I can use only Gemini API.
dajonker 1y ago
Try MinerU 2.5 with two-step parsing. It gives good results with bounding boxes per block. Not sure if you can get it to do more detailed such as word or character level.

KoolKat23 1y ago

These days it does just that, it'll say null or whatever if you give it the option. When it does make it up, it tends to be limitation of the image qualify ( max dpi). Blotchy text and specific typeface make 6's look like 8's, even to the non-discerning eye, a human would think it's an 8, zoom in and see it's a 6. Google's image quality on uploads is still streets ahead of openai for instance btw.