4 ms·

Gemini is still my top choice within production software for typical data extraction from unstructured data. Gemini Flash Lite feels like a cheat code for speed

by bicx 2mo ago

Gemini is still my top choice within production software for typical data extraction from unstructured data. Gemini Flash Lite feels like a cheat code for speed, and it's really cheap. Some other Chinese models are also fast and cheap, but a harder sell in a U.S. production environment.

msp26 2mo ago
[dead]
MrBuddyCasino 2mo ago
Yeah Gemini 3.5 Flash Lite is really good. Which Chinese models can you recommend?
- b345 2mo ago
  I've been using Qwen3.5-9B, hosted locally for PDF data extraction and it performs pretty well when extracting data from tables and infographics
- SkalskiP 2mo ago
  Hi, I’m the author of this blog. It depends on how strong of a model you need, but in general, Qwen is easily the best among the Chinese models right now. Over the last two weeks, Qwen released two new models. Qwen3.8-Max is totally insane, but it’s only available through the Alibaba Cloud API. I wrote a similar blog covering Qwen3.8-Max: [https://blog.roboflow.com/qwen3-8-max/ https://blog.roboflow.com/qwen3-8-max/](https://blog.roboflow.com/qwen3-8-max/ https://blog.roboflow.com/qwen3-8-max/) If you’re looking for something you can run locally, Qwen3.8-27B might be a great option. On Friday, I did a quick comparison between Qwen3.8-Max and Qwen3.8-27B: [https://x.com/skalskip92/status/2088411215441621469?s=20 https://x.com/skalskip92/status/2088411215441621469?s=20](https://x.com/skalskip92/status/2088411215441621469?s=20 https://x.com/skalskip92/status/2088411215441621469?s=20)
  - kanemcgrath 2mo ago
    Googles local gemma models which target roughly the same parameter count range, are known for being a lot better at vision tasks than qwen, no idea if 3.8 has changed that though
    - SkalskiP 2mo ago
      Really? Gemma4-31B should be better than Qwen3.8-27B? I'm happy to test that.
ComputerGuru 2mo ago
Speaking from experience here, flash lite models have amazing price, speed, and perform far above their size, but are susceptible to very bad instruction following and recall when either complexity or context size inch up. They’ll just forget to apply your instructions to portions of the input, and repeat parts of the input that should be returned verbatim as direct quotes but with subtle changes (breaking urls, for example).
- MrBuddyCasino 2mo ago
  Yes you have to continuously tune the prompts ever so subtly. 3.5 is a lot better than than 3.1 tho. Important to remember that json schema instructions take precedence over the normal prompt, so move as much into property descriptions as possible.
  - ComputerGuru 2mo ago
    This was 3.5 flash lite, actually, and after prompt tuning. It was very clearly an issue that correlated with input (JSON array) size, the more elements in the batch, the higher the error rate. 3.0 flash (not lite) handled it like a champ though, fwiw.

by bicx 2mo ago

msp26 2mo ago

[dead]

MrBuddyCasino 2mo ago

Yeah Gemini 3.5 Flash Lite is really good. Which Chinese models can you recommend?

b345 2mo ago
I've been using Qwen3.5-9B, hosted locally for PDF data extraction and it performs pretty well when extracting data from tables and infographics
SkalskiP 2mo ago
Hi, I’m the author of this blog. It depends on how strong of a model you need, but in general, Qwen is easily the best among the Chinese models right now. Over the last two weeks, Qwen released two new models. Qwen3.8-Max is totally insane, but it’s only available through the Alibaba Cloud API. I wrote a similar blog covering Qwen3.8-Max: [https://blog.roboflow.com/qwen3-8-max/ https://blog.roboflow.com/qwen3-8-max/](https://blog.roboflow.com/qwen3-8-max/ https://blog.roboflow.com/qwen3-8-max/) If you’re looking for something you can run locally, Qwen3.8-27B might be a great option. On Friday, I did a quick comparison between Qwen3.8-Max and Qwen3.8-27B: [https://x.com/skalskip92/status/2088411215441621469?s=20 https://x.com/skalskip92/status/2088411215441621469?s=20](https://x.com/skalskip92/status/2088411215441621469?s=20 https://x.com/skalskip92/status/2088411215441621469?s=20)
- kanemcgrath 2mo ago
  Googles local gemma models which target roughly the same parameter count range, are known for being a lot better at vision tasks than qwen, no idea if 3.8 has changed that though
  - SkalskiP 2mo ago
    Really? Gemma4-31B should be better than Qwen3.8-27B? I'm happy to test that.

ComputerGuru 2mo ago

Speaking from experience here, flash lite models have amazing price, speed, and perform far above their size, but are susceptible to very bad instruction following and recall when either complexity or context size inch up. They’ll just forget to apply your instructions to portions of the input, and repeat parts of the input that should be returned verbatim as direct quotes but with subtle changes (breaking urls, for example).

MrBuddyCasino 2mo ago
Yes you have to continuously tune the prompts ever so subtly. 3.5 is a lot better than than 3.1 tho. Important to remember that json schema instructions take precedence over the normal prompt, so move as much into property descriptions as possible.
- ComputerGuru 2mo ago
  This was 3.5 flash lite, actually, and after prompt tuning. It was very clearly an issue that correlated with input (JSON array) size, the more elements in the batch, the higher the error rate. 3.0 flash (not lite) handled it like a champ though, fwiw.