4 ms·

were you guys able to finish running the benchmark with mistral and got a 70% score? Missed that Edit - I see it on the Benchmark page now. Woof, low 70% score

• themanmaran 2y ago
  Yup, surprising results! We were able to dig in a bit more. Main culprit is the overzealous "image extraction". Where if Mistral classifies something as an image, it will replace the entire section with (image)[image_002). And it happened with a lot of full documents as well. Ex: most receipts got classified as images, and so it didn't extract any text.
  • lingjiekong 2y ago
    where do you find this regarding "Where if Mistral classifies something as an image, it will replace the entire section with (image)[image_002)."?
    • culi 2y ago
      themanmaran works at Omni so presumably they have access to the actual resulting data from this study
  • cdolan 2y ago
    This sounds like a real problem and hurdle for North American (US/CAN in particular) invoice and receipt processing?