5 ms·

Hi, I’m the author of this blog post. I wrote it about 4 weeks ago, and the VLM world is moving so fast that it’s already kinda outdated. I think Gemini 3.7 Fla

Hi, I’m the author of this blog post. I wrote it about 4 weeks ago, and the VLM world is moving so fast that it’s already kinda outdated. I think Gemini 3.7 Flash might be a better choice now, especially when you factor in the price. Here’s a comparison of the best low-cost models I put together last week. What’s crazy is that Gemini 3.7 Flash is now 50% off on OpenRouter, and this chart doesn’t even account for that discount. https://x.com/skalskip92/status/2088032652301304121?s=20 https://x.com/skalskip92/status/2088032652301304121?s=20

MostlyStable 2mo ago
Curious why you didn't try Gemini 3 pro? That is the model I've been using for OCR entry of handwritten datasheets (JPGS of datasheets, structured JSON output). At my scale, the cost of 3 pro is basically not an issue, but if there are improvements in quality, I'd definitely be willing to explore other models
- bastawhiz 2mo ago
  3 Pro is quickly approaching one year old. There's almost no reason to benchmark it, especially since a new version of Gemini Pro was supposed to be released mid 2026 and hasn't seen the light of day.
  - MostlyStable 2mo ago
    That would make sense if we already knew that, for these kinds of tasks it was significantly worse. The tests that I'm aware of for these tasks show it as still performing near the top.
  - tziki 2mo ago
    I think it definitely makes sense since it's still the best Google has to offer in the "pro" tier.
    - bastawhiz 2mo ago
      3 and 3.1 Pro are both marked as deprecated by Google. Even if they're the best Google offers, it would be foolish to choose a model that's explicitly deprecated. It's not a technical problem, it's a commercial one. If Google can't ship a model to replace the one they deprecated, that tells you everything you need to know about choosing a Gemini model for whatever you're trying to do.
      - heaney-555 2mo ago
        3.1 Pro is not deprecated!
        bastawhiz 2mo ago
        https://ai.google.dev/gemini-api/docs/deprecations https://ai.google.dev/gemini-api/docs/deprecations That link shows 3.1 pro listed as deprecated with no replacement model.
        qlte 2mo ago
        No, that's the "preview" version (gemini-3.1-pro-preview) aka the beta/early version before the official release of 3.1 Pro.
        deleted 1mo ago
        [deleted]
        heaney-555 1mo ago
        You are misreading that link. That's a list of all their models. Just like 3.7 Flash, 3.1 Pro says "No shutdown date announced."
- ppaanngggg 2mo ago
  [dead]
- gdudeman 2mo ago
  In my experience starting with Gemini 2.5 Pro, moving to 3 and 3.1, 3.5 Flash, 3.6 Flash, and finally 3.7 Flash, 3.7 Flash is just as good if not better than 3 especially on high resolution mode (same token count per page as 3.1). I run complicated, messy PDFs through these models. 2.5 Pro required a lot of kludgy hacks to get it to fully "see," but from 3.1 pro on I've removed many of them and haven't spotted problems. 3.7 Flash scores better than 3.1 pro on most benchmarks, leading me to believe that even if your OCR requires reasoning to interpret text or data, 3.7 Flash is probably going to be better.
- yieldcrv 2mo ago
  The “pro” moniker means nothing these models aren’t successors and barely have a common ancestor, they are independently baked in the training oven and assigned a semantic version randomly by someone trying to show initiative but not trying to do on the toes of the last guy who got promoted first So 3 pro is outdated and will likely never exit preview The “flash” and “lite” models are the real “pro” in colloquial ideas of fleshed out and capability, at this point. they’re better, faster and cheaper, larger context windows keeping up with the industry and more
  - heaney-555 2mo ago
    They are smaller models, and you can tell. Small models make dumb common-sense mistakes that big models never do. This is the "smell" many talk about.
    - sidibe 2mo ago
      Do you have cases where you still see 3.1 pro outperforming 3.7 flash?
      - heaney-555 2mo ago
        Yes, for complex questions of biology, physics, and analysis of anomalies. 3.7 Flash is better at coding, sure, but AI is not just for coding.
    - yieldcrv 2mo ago
      hasn't been an issue since 3.5 for me, what have you seen, say, in the last two months
      - heaney-555 2mo ago
        For complex questions of biology, physics, and analysis of anomalies, 3.1 Pro is still better than 3.7 Flash for me. 3.7 Flash is better at coding, sure, but AI is not just for coding.
Melatonic 2mo ago
What about Gemma ?
runtime_lens 2mo ago
[flagged]

MostlyStable 2mo ago

Curious why you didn't try Gemini 3 pro? That is the model I've been using for OCR entry of handwritten datasheets (JPGS of datasheets, structured JSON output). At my scale, the cost of 3 pro is basically not an issue, but if there are improvements in quality, I'd definitely be willing to explore other models

bastawhiz 2mo ago
3 Pro is quickly approaching one year old. There's almost no reason to benchmark it, especially since a new version of Gemini Pro was supposed to be released mid 2026 and hasn't seen the light of day.
- MostlyStable 2mo ago
  That would make sense if we already knew that, for these kinds of tasks it was significantly worse. The tests that I'm aware of for these tasks show it as still performing near the top.
- tziki 2mo ago
  I think it definitely makes sense since it's still the best Google has to offer in the "pro" tier.
  - bastawhiz 2mo ago
    3 and 3.1 Pro are both marked as deprecated by Google. Even if they're the best Google offers, it would be foolish to choose a model that's explicitly deprecated. It's not a technical problem, it's a commercial one. If Google can't ship a model to replace the one they deprecated, that tells you everything you need to know about choosing a Gemini model for whatever you're trying to do.
    - heaney-555 2mo ago
      3.1 Pro is not deprecated!
      - bastawhiz 2mo ago
        https://ai.google.dev/gemini-api/docs/deprecations https://ai.google.dev/gemini-api/docs/deprecations That link shows 3.1 pro listed as deprecated with no replacement model.
        qlte 2mo ago
        No, that's the "preview" version (gemini-3.1-pro-preview) aka the beta/early version before the official release of 3.1 Pro.
        deleted 1mo ago
        [deleted]
        heaney-555 1mo ago
        You are misreading that link. That's a list of all their models. Just like 3.7 Flash, 3.1 Pro says "No shutdown date announced."
ppaanngggg 2mo ago
[dead]
gdudeman 2mo ago
In my experience starting with Gemini 2.5 Pro, moving to 3 and 3.1, 3.5 Flash, 3.6 Flash, and finally 3.7 Flash, 3.7 Flash is just as good if not better than 3 especially on high resolution mode (same token count per page as 3.1). I run complicated, messy PDFs through these models. 2.5 Pro required a lot of kludgy hacks to get it to fully "see," but from 3.1 pro on I've removed many of them and haven't spotted problems. 3.7 Flash scores better than 3.1 pro on most benchmarks, leading me to believe that even if your OCR requires reasoning to interpret text or data, 3.7 Flash is probably going to be better.
yieldcrv 2mo ago
The “pro” moniker means nothing these models aren’t successors and barely have a common ancestor, they are independently baked in the training oven and assigned a semantic version randomly by someone trying to show initiative but not trying to do on the toes of the last guy who got promoted first So 3 pro is outdated and will likely never exit preview The “flash” and “lite” models are the real “pro” in colloquial ideas of fleshed out and capability, at this point. they’re better, faster and cheaper, larger context windows keeping up with the industry and more
- heaney-555 2mo ago
  They are smaller models, and you can tell. Small models make dumb common-sense mistakes that big models never do. This is the "smell" many talk about.
  - sidibe 2mo ago
    Do you have cases where you still see 3.1 pro outperforming 3.7 flash?
    - heaney-555 2mo ago
      Yes, for complex questions of biology, physics, and analysis of anomalies. 3.7 Flash is better at coding, sure, but AI is not just for coding.
  - yieldcrv 2mo ago
    hasn't been an issue since 3.5 for me, what have you seen, say, in the last two months
    - heaney-555 2mo ago
      For complex questions of biology, physics, and analysis of anomalies, 3.1 Pro is still better than 3.7 Flash for me. 3.7 Flash is better at coding, sure, but AI is not just for coding.

Melatonic 2mo ago

What about Gemma ?

runtime_lens 2mo ago

[flagged]