3 ms·

I find it challenging to accept something that talks about "OCR" but then I upload a PDF with text in images, and when I query the document after upload, I get

by sbarre 2y ago

I find it challenging to accept something that talks about "OCR" but then I upload a PDF with text in images, and when I query the document after upload, I get a message that says "I can't interpret images".. Then are you actually doing OCR, or are you just extracting embedded text?

deleted 2y ago
[deleted]
0x62 2y ago
I’d imagine their capabilities mirror that of Mistral OCR [1]. Mistral outputs markdown, the image would have to be convertible to a reasonably useful markdown structure (charts, tables etc). [1] https://mistral.ai/en/news/mistral-ocr https://mistral.ai/en/news/mistral-ocr
- themanmaran 2y ago
  This highlights the biggest issue I've found with Mistral OCR. Many of the documents I upload are entirely classified as images, which means no OCR is being run. Pretty much anything with a different colored background gets returned as (image)[image_001]. Example: https://omni-demo-data.s3.us-east-1.amazonaws.com/test/1739882935975_Free+to+use+Biogas+PDF.pdf https://omni-demo-data.s3.us-east-1.amazonaws.com/test/17398...
  - mtillman 2y ago
    LLMs tend to be a hammer in search of a nail when it comes to documents that have imagery. We decided on CV models which results in a high 90s midpoint for the docs our customers care about. If you can afford to go with a cv pipeline, it can outperform all of the LLMs by some margin.
- bilater 2y ago
  Yes - unfortunately it seems they don't read images in the pdf.
tmpz22 2y ago
Same vein - YouTube most (all?) llm integrations just scrape the transcript. I -think- google's aistudio does more but I'm unsure. I mean I get it bulk video processing would be crazy expensive, but at least mention you're only analyzing the transcript especially if you're a paid product.
- wildzzz 2y ago
  Whisper does do text to speech but yes, nearly all just read off the subtitles. There's a video by f4mi on YouTube where she tricked the summarizing bots with off-screen captions filled with nonsense.

by sbarre 2y ago

deleted 2y ago

[deleted]

0x62 2y ago

I’d imagine their capabilities mirror that of Mistral OCR [1]. Mistral outputs markdown, the image would have to be convertible to a reasonably useful markdown structure (charts, tables etc). [1] https://mistral.ai/en/news/mistral-ocr https://mistral.ai/en/news/mistral-ocr

themanmaran 2y ago
This highlights the biggest issue I've found with Mistral OCR. Many of the documents I upload are entirely classified as images, which means no OCR is being run. Pretty much anything with a different colored background gets returned as (image)[image_001]. Example: https://omni-demo-data.s3.us-east-1.amazonaws.com/test/1739882935975_Free+to+use+Biogas+PDF.pdf https://omni-demo-data.s3.us-east-1.amazonaws.com/test/17398...
- mtillman 2y ago
  LLMs tend to be a hammer in search of a nail when it comes to documents that have imagery. We decided on CV models which results in a high 90s midpoint for the docs our customers care about. If you can afford to go with a cv pipeline, it can outperform all of the LLMs by some margin.
bilater 2y ago
Yes - unfortunately it seems they don't read images in the pdf.

tmpz22 2y ago

Same vein - YouTube most (all?) llm integrations just scrape the transcript. I -think- google's aistudio does more but I'm unsure. I mean I get it bulk video processing would be crazy expensive, but at least mention you're only analyzing the transcript especially if you're a paid product.

wildzzz 2y ago
Whisper does do text to speech but yes, nearly all just read off the subtitles. There's a video by f4mi on YouTube where she tricked the summarizing bots with off-screen captions filled with nonsense.