4 ms·

News update: OCR company touts new benchmark that shows its own products are the most performant.

jauntywundrkind 2y ago
I searched for any link between OmniAI and Alibaba's Qwen, but I can't find any link. Do you know anything I don't know? All of these models are open source (I think?). They could presumably build their work on any of these options. It behooves them to pick well. And establish some authority along the way.
- rustc 2y ago
  The model with the best accuracy in the linked benchmark is "OmniAI" (OP's company) which looks like a paid model, not open source [1]. [1]: https://getomni.ai/pricing https://getomni.ai/pricing
johnisgood 2y ago
Someone should try to reproduce and post it here. I can't, my PC is about 15 years old. :( (It is not a joke.)
- rustc 2y ago
  Reproducing the whole benchmark would be expensive, OmniAi starts at $250/month.
  - themanmaran 2y ago
    Generally running the whole benchmark is ~$200, since all the providers cost money. But if anyone wants to specifically benchmark Omni just drop us a note and we'll make the credits available.
  - johnisgood 2y ago
    So not all of them are local and open source? Ugh.
    - qingcharles 2y ago
      I don't see why you couldn't run any of those locally if you buy the right hardware?
      - johnisgood 2y ago
        I haven't checked myself, so I'm not sure, others might be able to provide the answer though. If they (all of the mentioned ones) are open source and can be ran locally, then most likely, yes. From what I remember, they are all local and open source, so the answer is yes, if I am correct.
      - ipsum2 2y ago
        Mistral ocr is closed source
        johnisgood 2y ago
        Thanks!
kapitalx 2y ago
To be fair, they didn't include themselves at all in the graph.
- azinman2 2y ago
  They did. It’s in the #1 spot Update: looks like the removed themselves from the graph since I saw it earlier today!
  - axpy906 2y ago
    Yup, they did. The beauty of version control: https://github.com/getomni-ai/benchmark/commit/0544e2a4394239502db00226805519e62b1370a2 https://github.com/getomni-ai/benchmark/commit/0544e2a439423...

I searched for any link between OmniAI and Alibaba's Qwen, but I can't find any link. Do you know anything I don't know? All of these models are open source (I think?). They could presumably build their work on any of these options. It behooves them to pick well. And establish some authority along the way.

rustc 2y ago
The model with the best accuracy in the linked benchmark is "OmniAI" (OP's company) which looks like a paid model, not open source [1]. [1]: https://getomni.ai/pricing https://getomni.ai/pricing

johnisgood 2y ago

Someone should try to reproduce and post it here. I can't, my PC is about 15 years old. :( (It is not a joke.)

rustc 2y ago
Reproducing the whole benchmark would be expensive, OmniAi starts at $250/month.
- themanmaran 2y ago
  Generally running the whole benchmark is ~$200, since all the providers cost money. But if anyone wants to specifically benchmark Omni just drop us a note and we'll make the credits available.
- johnisgood 2y ago
  So not all of them are local and open source? Ugh.
  - qingcharles 2y ago
    I don't see why you couldn't run any of those locally if you buy the right hardware?
    - johnisgood 2y ago
      I haven't checked myself, so I'm not sure, others might be able to provide the answer though. If they (all of the mentioned ones) are open source and can be ran locally, then most likely, yes. From what I remember, they are all local and open source, so the answer is yes, if I am correct.
    - ipsum2 2y ago
      Mistral ocr is closed source
      - johnisgood 2y ago
        Thanks!

kapitalx 2y ago

To be fair, they didn't include themselves at all in the graph.

azinman2 2y ago
They did. It’s in the #1 spot Update: looks like the removed themselves from the graph since I saw it earlier today!
- axpy906 2y ago
  Yup, they did. The beauty of version control: https://github.com/getomni-ai/benchmark/commit/0544e2a4394239502db00226805519e62b1370a2 https://github.com/getomni-ai/benchmark/commit/0544e2a439423...