6 ms·

Mistral looks set to challenge AI frontrunners Google and OpenAI

79 points by jmsflknr 3y ago

ls612 3y ago
Is Mistral only doing LLMs? Are they going to for instance make a foundation model to compete with DALL-E/Stable Diffusion/Midjourney?
andy99 3y ago
This is interesting in the context of Meta blowing billions of dollars just now on compute in order to buy their way into competitiveness. It seems so far like raw $ is only very loosely correlated with the ability to come out with leading models. See also UAE and the Falcons. I wonder if there's any sign of that changing and big money making more of a difference, or if we'll see compute revert back to purely a commodity and "fecundity" becoming the ruling factor. Edit: finally archived https://archive.md/RrXWo https://archive.md/RrXWo
- nightski 3y ago
  That is a really interesting alternative history you have there...
dontupvoteme 3y ago
Mistral IMO: 1. Doesn't have to pay the obscene alignment tax that OpenAI/etc have to (something between 20-50% of from what i understand) nor worry nearly as much about """the brand""" and the baggage which comes with that. 2. Likely will have/has the EU behind them out of pragmatic protectionism realpolitik verus the US (which basically started with GDPR) - though it might end up being a duo with Aleph Alpha. 3. Generally has the widespread support of the hobbyist/opensource community (for whatever aims those may be), both out of ethical/moral consideration and performance/quality reasons 4. Seems already *quite* competitive with GPT3. I rarely find i need to invoke GPT4 and when I have to, I'm annoyed with the latency and milquetoastian nature of the damn thing. Wish I could purchase stock in these guys honestly
- ghostly_s 3y ago
  "Alignment tax?"
  - Spivak 3y ago
    If you want an AI to actually do things like answer your questions instead of just complete the question you have to nudge the model to do that which makes it stray from the "best" answer. There's a balance you need to strike between aligning it enough so that it gives answers humans actually want and not aligning too much to degrade the models performance. People have an axe to grind with OpenAI because they purposely align their model to be boring as the intended audience is companies embedding it in their own products and try to fight the bias inherent in a model trained on the writings of the average chronically online Redditor. The frustration isn't unfounded, if you want to use the model for programming having to pay the 'alignment tax' because someone else is using the model for a customer service bot and can't match energy with an angry customer sucks.
  - _giorgio_ 3y ago
    alignment tax = Training needed to cancel bad behaviour, political incorrectness, dangerous results etc.
    - deleted 3y ago
      [deleted]
    - jjoonathan 3y ago
      Yes, but you want alignment even if you don't want censorship. A very intelligent but unaligned model will be prone to doing useless things like "auto-complete" your question into a more elaborate version or responding with "just google it dumbass" and other forms of internet vitriol.
      - declaredapple 3y ago
        > Yes, but you want alignment even if you don't want censorship. Instruction tuning helps a lot with this but what a lot of people mean is the refusal to do things. You get to chose how "aligned" it is, for some usecases like talking to customers you definitely want something very "safe" (won't start using slurs or something terrible). But for direct usage you generally never want it to refuse to do anything. Checkout Anthropic on the extreme side - every iteration of Claude has gotten worse on the chatbot arena (elo based on humans blindly comparing responses). https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
foolfoolz 3y ago
mistral will be successful as a matter of national security for the EU. it literally cannot fail at this point
- ben_w 3y ago
  > it literally cannot fail at this point Ahh, the magic words that cause any venture to fail
  - outside1234 3y ago
    Or take all of us down such that they don't fail (see Bank bailouts)
- kkzz99 3y ago
  Mistrals biggest risk factor is the EU. The overzealous regulation could be stifling innovation, especially open source.
neonate 3y ago
https://archive.ph/RrXWo https://archive.ph/RrXWo
jfim 3y ago
Is Google actually a frontrunner in this race? I remember Bard was pretty unimpressive when it came out, is it improving fast enough to be competitive?
- declaredapple 3y ago
  Their currently available offerings are "meh", but they're definitely top 5 (Currently they're number 4). This leaderboard is the only one most people trust as it uses an ELO system with humans blindly comparing and rating them in the chatbot arena. https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Current leaders are OpenAI, Anthropic, Mistral, Google, 01-ai, and then various Llama finetunes and Llama itself (Meta)
  - Jackson__ 3y ago
    This leaderboard is so funny when you look at Anthropic's Claude. Every version After 1.0 gets a worse score. It seems their vision of "alignment" does not align with the users vision of a competent assistant even remotely.
    - declaredapple 3y ago
      Yeah it's mystifying to me that they because the #1 commercial OpenAI competitor and then decided to go all "Helpless and Harmless"
    - frognumber 3y ago
      I don't think that's their business model. I would never use Claude for personal use. However, my employer wants to make sure nothing reputation-harming happens. That's a lot more important than model quality; everything is light years ahead of where we were four years ago. For a lot of applications which face customers / citizens / students / ..., NEVER screwing up is a lot more important than quality. For my own use, I prefer interacting with soul, humanity, attitude, and edge. For my employer's use, it's different. I think there's space for both. As an investor, I'd be bullish on both Anthropic and something with no safety built in. I'd be a lot less bullish on what's in between.
Jackson__ 3y ago
Mistral, in my opinion, looks to be the most exciting company in the LLM space right now; judging by their pace of releases alone. GPT3 to 4 took almost 3 years. Google Gemini took around 3/4 of a year. Mistral 7b to Mixtral 8x7b and simultaneously Mixtral Medium was ~82 days. That's crazy fast.
Semaphor 3y ago
I realize that by using them for entertainment, I'm not exactly a normal user, but Mistral has been such a breath of fresh air. With OpenAI it was always fighting against the model, making it useable despite all the "I can't do that Dave". Mistral just works.
- dontupvoteme 3y ago
  if by entertainment you mean roleplay, that's actually seemingly a huge market which is not served due to reasons. A lot of women seem to be interested in it too, maybe even more than men.
  - Semaphor 3y ago
    In my case, it’s actually using it as a bot one can talk to in two twitch chats I mod. One is a foodtruck, which works well enough with OpenAI, the other one is a blue plushie penis called Scomo, and that’s the one where OpenAI is difficult ;)
    - thelastparadise 3y ago
      these
    - dontupvoteme 3y ago
      Oh that's cool! I actually was playing around with shadowmodding a bit. When you have to parse some of the worst messages you've ever (potentially) seen it's helpful to have an AI not reject your request.
tomaszs 3y ago
Don't use the linked archive.ph, the page contains copyright infringements

79 points by jmsflknr 3y ago

Is Mistral only doing LLMs? Are they going to for instance make a foundation model to compete with DALL-E/Stable Diffusion/Midjourney?

andy99 3y ago

This is interesting in the context of Meta blowing billions of dollars just now on compute in order to buy their way into competitiveness. It seems so far like raw $ is only very loosely correlated with the ability to come out with leading models. See also UAE and the Falcons. I wonder if there's any sign of that changing and big money making more of a difference, or if we'll see compute revert back to purely a commodity and "fecundity" becoming the ruling factor. Edit: finally archived https://archive.md/RrXWo https://archive.md/RrXWo

nightski 3y ago
That is a really interesting alternative history you have there...

dontupvoteme 3y ago

Mistral IMO: 1. Doesn't have to pay the obscene alignment tax that OpenAI/etc have to (something between 20-50% of from what i understand) nor worry nearly as much about """the brand""" and the baggage which comes with that. 2. Likely will have/has the EU behind them out of pragmatic protectionism realpolitik verus the US (which basically started with GDPR) - though it might end up being a duo with Aleph Alpha. 3. Generally has the widespread support of the hobbyist/opensource community (for whatever aims those may be), both out of ethical/moral consideration and performance/quality reasons 4. Seems already *quite* competitive with GPT3. I rarely find i need to invoke GPT4 and when I have to, I'm annoyed with the latency and milquetoastian nature of the damn thing. Wish I could purchase stock in these guys honestly

ghostly_s 3y ago
"Alignment tax?"
- Spivak 3y ago
  If you want an AI to actually do things like answer your questions instead of just complete the question you have to nudge the model to do that which makes it stray from the "best" answer. There's a balance you need to strike between aligning it enough so that it gives answers humans actually want and not aligning too much to degrade the models performance. People have an axe to grind with OpenAI because they purposely align their model to be boring as the intended audience is companies embedding it in their own products and try to fight the bias inherent in a model trained on the writings of the average chronically online Redditor. The frustration isn't unfounded, if you want to use the model for programming having to pay the 'alignment tax' because someone else is using the model for a customer service bot and can't match energy with an angry customer sucks.
- _giorgio_ 3y ago
  alignment tax = Training needed to cancel bad behaviour, political incorrectness, dangerous results etc.
  - deleted 3y ago
    [deleted]
  - jjoonathan 3y ago
    Yes, but you want alignment even if you don't want censorship. A very intelligent but unaligned model will be prone to doing useless things like "auto-complete" your question into a more elaborate version or responding with "just google it dumbass" and other forms of internet vitriol.
    - declaredapple 3y ago
      > Yes, but you want alignment even if you don't want censorship. Instruction tuning helps a lot with this but what a lot of people mean is the refusal to do things. You get to chose how "aligned" it is, for some usecases like talking to customers you definitely want something very "safe" (won't start using slurs or something terrible). But for direct usage you generally never want it to refuse to do anything. Checkout Anthropic on the extreme side - every iteration of Claude has gotten worse on the chatbot arena (elo based on humans blindly comparing responses). https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

foolfoolz 3y ago

mistral will be successful as a matter of national security for the EU. it literally cannot fail at this point

ben_w 3y ago
> it literally cannot fail at this point Ahh, the magic words that cause any venture to fail
- outside1234 3y ago
  Or take all of us down such that they don't fail (see Bank bailouts)
kkzz99 3y ago
Mistrals biggest risk factor is the EU. The overzealous regulation could be stifling innovation, especially open source.

neonate 3y ago

https://archive.ph/RrXWo https://archive.ph/RrXWo

jfim 3y ago

Is Google actually a frontrunner in this race? I remember Bard was pretty unimpressive when it came out, is it improving fast enough to be competitive?

declaredapple 3y ago
Their currently available offerings are "meh", but they're definitely top 5 (Currently they're number 4). This leaderboard is the only one most people trust as it uses an ELO system with humans blindly comparing and rating them in the chatbot arena. https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Current leaders are OpenAI, Anthropic, Mistral, Google, 01-ai, and then various Llama finetunes and Llama itself (Meta)
- Jackson__ 3y ago
  This leaderboard is so funny when you look at Anthropic's Claude. Every version After 1.0 gets a worse score. It seems their vision of "alignment" does not align with the users vision of a competent assistant even remotely.
  - declaredapple 3y ago
    Yeah it's mystifying to me that they because the #1 commercial OpenAI competitor and then decided to go all "Helpless and Harmless"
  - frognumber 3y ago
    I don't think that's their business model. I would never use Claude for personal use. However, my employer wants to make sure nothing reputation-harming happens. That's a lot more important than model quality; everything is light years ahead of where we were four years ago. For a lot of applications which face customers / citizens / students / ..., NEVER screwing up is a lot more important than quality. For my own use, I prefer interacting with soul, humanity, attitude, and edge. For my employer's use, it's different. I think there's space for both. As an investor, I'd be bullish on both Anthropic and something with no safety built in. I'd be a lot less bullish on what's in between.

Jackson__ 3y ago

Mistral, in my opinion, looks to be the most exciting company in the LLM space right now; judging by their pace of releases alone. GPT3 to 4 took almost 3 years. Google Gemini took around 3/4 of a year. Mistral 7b to Mixtral 8x7b and simultaneously Mixtral Medium was ~82 days. That's crazy fast.

Semaphor 3y ago

I realize that by using them for entertainment, I'm not exactly a normal user, but Mistral has been such a breath of fresh air. With OpenAI it was always fighting against the model, making it useable despite all the "I can't do that Dave". Mistral just works.

dontupvoteme 3y ago
if by entertainment you mean roleplay, that's actually seemingly a huge market which is not served due to reasons. A lot of women seem to be interested in it too, maybe even more than men.
- Semaphor 3y ago
  In my case, it’s actually using it as a bot one can talk to in two twitch chats I mod. One is a foodtruck, which works well enough with OpenAI, the other one is a blue plushie penis called Scomo, and that’s the one where OpenAI is difficult ;)
  - thelastparadise 3y ago
    these
  - dontupvoteme 3y ago
    Oh that's cool! I actually was playing around with shadowmodding a bit. When you have to parse some of the worst messages you've ever (potentially) seen it's helpful to have an AI not reject your request.

tomaszs 3y ago

Don't use the linked archive.ph, the page contains copyright infringements