4 ms·

Hmm, why was their starting point not something like BERT: * already known as SotA for text classification and similarity back in 2023 * natively mul

Hmm, why was their starting point not something like BERT: * already known as SotA for text classification and similarity back in 2023 * natively multi-lingual

embedding-shape 11mo ago
People generally sleep when you start talking about fine-tuned BERT and CLIP, although they do a fairly decent job as long as you have good data and know what you're doing. But no, they want to pay $0.1 per request to recognize if a photo has a person in it by asking a multimodal LLM deployed across 8x GPUs, for some reason, instead of just spending some hours with CLIP and run it effectively even on CPU.
- efavdb 11mo ago
  Are you suggesting use the clip embedding for the text as a feature to train a standard Ml model on?
  - PaulHoule 11mo ago
    I think he is. I do things like that plenty.
  - daemonologist 11mo ago
    I think they're suggesting doing that with BERT for text and CLIP for images. Which in my experience is indeed quite effective (and easy/fast). There have been some developments in the image-of-text/other-than-photograph area though recently. From Meta (although they seem unsure of what exactly their AI division is called): https://arxiv.org/abs/2510.05014 https://arxiv.org/abs/2510.05014 and Qihoo360: https://arxiv.org/abs/2510.27350 https://arxiv.org/abs/2510.27350 for instance.
- cestith 11mo ago
  I’m no ML engineer and far from an LLM expert. Just reading the article though it seemed to me that leveraging an SQL database here was a bigger issue than using traditional ML on the data, rather than the LLM being a win specifically. Just finding anything that was better suited than string matching on a RDBMS to the type of inputs seems like the natural conclusion when the complaint in the article itself was literally about SQL.
- Aurornis 11mo ago
  > they do a fairly decent job as long as you have good data and know what you're doing. This is the bottleneck in my experience. Going for the expensive per-request LLM gets something shipped now that you can wow the execs with. Setting up a whole process to gather and annotate data, train models, run evals, and iterate takes time. The execs who hired those expensive AI engineers want their results right now, not after a process of hiring more people to collect and annotate the data.
- keeda 11mo ago
  >... as long as you have good data and know what you're doing. I think you've just identified, in a set-theoretic complementary manner, the TAM for GenAI.
  - throwaway314155 11mo ago
    What's TAM?
    - keeda 11mo ago
      https://en.wikipedia.org/wiki/Total_addressable_market https://en.wikipedia.org/wiki/Total_addressable_market
elzbardico 11mo ago
Because Enterprise Development only moves forward based on hyped technologies.
deepsquirrelnet 11mo ago
I love using encoder models, and they are generally a better technology for this kind of application. But the price of GPU instances is too damn high. I won’t lie that I’ve been unreasonably annoyed that I have to use a lot more compute than I need, for no other reason than an LLM API exists and it’s good enough in a relatively small throughput application.

Hmm, why was their starting point not something like BERT: * already known as SotA for text classification and similarity back in 2023 * natively multi-lingual

embedding-shape 11mo ago

People generally sleep when you start talking about fine-tuned BERT and CLIP, although they do a fairly decent job as long as you have good data and know what you're doing. But no, they want to pay $0.1 per request to recognize if a photo has a person in it by asking a multimodal LLM deployed across 8x GPUs, for some reason, instead of just spending some hours with CLIP and run it effectively even on CPU.

efavdb 11mo ago
Are you suggesting use the clip embedding for the text as a feature to train a standard Ml model on?
- PaulHoule 11mo ago
  I think he is. I do things like that plenty.
- daemonologist 11mo ago
  I think they're suggesting doing that with BERT for text and CLIP for images. Which in my experience is indeed quite effective (and easy/fast). There have been some developments in the image-of-text/other-than-photograph area though recently. From Meta (although they seem unsure of what exactly their AI division is called): https://arxiv.org/abs/2510.05014 https://arxiv.org/abs/2510.05014 and Qihoo360: https://arxiv.org/abs/2510.27350 https://arxiv.org/abs/2510.27350 for instance.
cestith 11mo ago
I’m no ML engineer and far from an LLM expert. Just reading the article though it seemed to me that leveraging an SQL database here was a bigger issue than using traditional ML on the data, rather than the LLM being a win specifically. Just finding anything that was better suited than string matching on a RDBMS to the type of inputs seems like the natural conclusion when the complaint in the article itself was literally about SQL.
Aurornis 11mo ago
> they do a fairly decent job as long as you have good data and know what you're doing. This is the bottleneck in my experience. Going for the expensive per-request LLM gets something shipped now that you can wow the execs with. Setting up a whole process to gather and annotate data, train models, run evals, and iterate takes time. The execs who hired those expensive AI engineers want their results right now, not after a process of hiring more people to collect and annotate the data.
keeda 11mo ago
>... as long as you have good data and know what you're doing. I think you've just identified, in a set-theoretic complementary manner, the TAM for GenAI.
- throwaway314155 11mo ago
  What's TAM?
  - keeda 11mo ago
    https://en.wikipedia.org/wiki/Total_addressable_market https://en.wikipedia.org/wiki/Total_addressable_market

elzbardico 11mo ago

Because Enterprise Development only moves forward based on hyped technologies.

deepsquirrelnet 11mo ago

I love using encoder models, and they are generally a better technology for this kind of application. But the price of GPU instances is too damn high. I won’t lie that I’ve been unreasonably annoyed that I have to use a lot more compute than I need, for no other reason than an LLM API exists and it’s good enough in a relatively small throughput application.