3 ms·

Because they distill

ctrlkctrls 4mo ago
it doesn't matter the reason. This is a race and nobody will care or remember how the winners got there. Mistral looks like it's fading away to irrelevance unless they can play alongside the similar sized models, or have some unique advantage other than being in Europe, for Europe. I was really excited for them back when they were startup that had the biggest European venture round ever. This space will have a few winners, and many losers. Google, plus either Anthropic or OpenAI most likely. Big models will see breakthroughs in inference performance/cost fall precipitously and small models will only exist on devices (Pixels and iPhones, cars, watches, bluetooth speakers, etc)
- gregorygoc 4mo ago
  It’s not that I don’t agree with you, I am just pointing out why it’s hard to catch up to scaling laws given the European economic (capital) and political (US would be upset if they found out Europeans distill) constraints. China is only bound by economic constraints.
  - KronisLV 4mo ago
    With the insight in your comment and this bit from the above one: > This is a race and nobody will care or remember how the winners got there. It seems like the EU should have paid China for the distillation datasets, esp. since Mistral isn’t even a governmental org.
- sofixa 4mo ago
  > This is a race and nobody will care or remember how the winners got there. For consumer AI, yes. For coding assistants, probably. For specific application "business" AI like the things Airbus announced the other day? Not at all. What matters for an Airbus using Mistral to build compliance documentation based on AI generated physics simulations is the enterprise relationship, reliability, compliance, forward deployed engineers helping with the fine tuning, quality, predictability, support. A Chinese lab having a better at benchmarks model that is cheaper is just irrelevant for that. And IMO, the real money in AI is this type of "business AI" deployment. Developer tooling tends to converge on becoming commoditised. Once you're a core supplier for a big bank and embedded in their processes, you're there untill you screw up with the pricing (like Broadcom), and even then.
k__ 4mo ago
Why doesn't Mistral distill?
- gregorygoc 4mo ago
  Good question, given that American companies basically threw copyright law into the trash, I think they should.
  - xienze 4mo ago
    Well if they did wouldn't be able to feel superior to Americans about that particular thing. Perish the thought!
    - __m 4mo ago
      It’s really a pity, why can’t they feel superior while breaking ToS and copyrights just like Americans can feel superior over Deepseek while breaking ToS and copyrights?
  - Saline9515 4mo ago
    American companies can't sur Chinese ones, but they can do it with European ones.
    - Matl 4mo ago
      So then the European ones should join with European copyright holders to sue OpenAI/Anthropic and watch them trying to BS their way around what they train on.
      - Saline9515 4mo ago
        Training a model on copyrighted material is fair use and copyright holders already operate a mafia-style extortion ring in Europe, so I don't think that's a good idea.
        4mo ago
opsnooperfax 4mo ago
I suppose losing with dignity is a consolation.
losvedir 4mo ago
I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right? Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.
- taneq 4mo ago
  Whether it's a 'problem' or not is viewpoint-dependent but it's against the OpenAI ToU: > You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: > [...] > * Use Output to develop models that compete with OpenAI. Source: https://openai.com/policies/row-terms-of-use/ https://openai.com/policies/row-terms-of-use/ (I'm also curious whether they consider developing a competing model to be illegal, or harmful, or abusive...?)
  - zhivota 4mo ago
    Feels like this should be some kind of anti-competitive violation even if it's not actually. Probably moot under this admin but still. It's like saying you can't use windows to develop an OS, or drive a Ford on the way to your job at Hyundai.
    - CamperBob2 4mo ago
      In heavily-unionized areas, you'd best not drive that Hyundai to your job at the Ford plant.
  - Matl 4mo ago
    > it's against the OpenAI ToU Given that OpenAI doesn't care about training on copyrighted data, why is suddenly their ToU something anyone should care about?
    - 4mo ago

it doesn't matter the reason. This is a race and nobody will care or remember how the winners got there. Mistral looks like it's fading away to irrelevance unless they can play alongside the similar sized models, or have some unique advantage other than being in Europe, for Europe. I was really excited for them back when they were startup that had the biggest European venture round ever. This space will have a few winners, and many losers. Google, plus either Anthropic or OpenAI most likely. Big models will see breakthroughs in inference performance/cost fall precipitously and small models will only exist on devices (Pixels and iPhones, cars, watches, bluetooth speakers, etc)

gregorygoc 4mo ago
It’s not that I don’t agree with you, I am just pointing out why it’s hard to catch up to scaling laws given the European economic (capital) and political (US would be upset if they found out Europeans distill) constraints. China is only bound by economic constraints.
- KronisLV 4mo ago
  With the insight in your comment and this bit from the above one: > This is a race and nobody will care or remember how the winners got there. It seems like the EU should have paid China for the distillation datasets, esp. since Mistral isn’t even a governmental org.
sofixa 4mo ago
> This is a race and nobody will care or remember how the winners got there. For consumer AI, yes. For coding assistants, probably. For specific application "business" AI like the things Airbus announced the other day? Not at all. What matters for an Airbus using Mistral to build compliance documentation based on AI generated physics simulations is the enterprise relationship, reliability, compliance, forward deployed engineers helping with the fine tuning, quality, predictability, support. A Chinese lab having a better at benchmarks model that is cheaper is just irrelevant for that. And IMO, the real money in AI is this type of "business AI" deployment. Developer tooling tends to converge on becoming commoditised. Once you're a core supplier for a big bank and embedded in their processes, you're there untill you screw up with the pricing (like Broadcom), and even then.

k__ 4mo ago

Why doesn't Mistral distill?

gregorygoc 4mo ago
Good question, given that American companies basically threw copyright law into the trash, I think they should.
- xienze 4mo ago
  Well if they did wouldn't be able to feel superior to Americans about that particular thing. Perish the thought!
  - __m 4mo ago
    It’s really a pity, why can’t they feel superior while breaking ToS and copyrights just like Americans can feel superior over Deepseek while breaking ToS and copyrights?
- Saline9515 4mo ago
  American companies can't sur Chinese ones, but they can do it with European ones.
  - Matl 4mo ago
    So then the European ones should join with European copyright holders to sue OpenAI/Anthropic and watch them trying to BS their way around what they train on.
    - Saline9515 4mo ago
      Training a model on copyrighted material is fair use and copyright holders already operate a mafia-style extortion ring in Europe, so I don't think that's a good idea.
      - 4mo ago

opsnooperfax 4mo ago

I suppose losing with dignity is a consolation.

losvedir 4mo ago

I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right? Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.

taneq 4mo ago
Whether it's a 'problem' or not is viewpoint-dependent but it's against the OpenAI ToU: > You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: > [...] > * Use Output to develop models that compete with OpenAI. Source: https://openai.com/policies/row-terms-of-use/ https://openai.com/policies/row-terms-of-use/ (I'm also curious whether they consider developing a competing model to be illegal, or harmful, or abusive...?)
- zhivota 4mo ago
  Feels like this should be some kind of anti-competitive violation even if it's not actually. Probably moot under this admin but still. It's like saying you can't use windows to develop an OS, or drive a Ford on the way to your job at Hyundai.
  - CamperBob2 4mo ago
    In heavily-unionized areas, you'd best not drive that Hyundai to your job at the Ford plant.
- Matl 4mo ago
  > it's against the OpenAI ToU Given that OpenAI doesn't care about training on copyrighted data, why is suddenly their ToU something anyone should care about?
  - 4mo ago