5 ms·

We ran some benchmarks comparing against Gemini Flash 2.0. You can find the full writeup here: https://reducto.ai/blog/lvm-ocr-accuracy-mistral-gemini https://r

We ran some benchmarks comparing against Gemini Flash 2.0. You can find the full writeup here: https://reducto.ai/blog/lvm-ocr-accuracy-mistral-gemini https://reducto.ai/blog/lvm-ocr-accuracy-mistral-gemini A high level summary is that while this is an impressive model, it underperforms even current SOTA VLMs on document parsing and has a tendency to hallucinate with OCR, table structure, and drop content.

hackernewds 2y ago
meanwhile, you're comparing it to the output of almost a trillion dollar company
- HaZeust 2y ago
  ... And? We're judging it for the merits of the technology it purports to be, not the pockets of the people that bankroll them. Probably not fair - sure, but when I pick my OCR, I want to pick SOTA. These comparisons and announcements help me find those.
- stann 2y ago
  The tagline boasts that it is "introducing the world’s best document understanding API". So, holding them to their marketing seems fair
  - neuronic 2y ago
    Isn't anyone who releases anything putting "the world's best blablabla" on their page nowadays? I've become entirely blind to it.
    - dwedge 2y ago
      If they put it, and it's subpar, I write off the product.
- raunakchowdhuri 2y ago
  comparisons to more outputs coming soon!
shrisukhani 2y ago
Anecdotally, we also found Gemini Flash to be better.