4 ms·

I'm surprised this isn't firmly attached to the top of HN right now for the entire day. This is a tiny company (appears 30 or so people?) that just scored a 2B

I'm surprised this isn't firmly attached to the top of HN right now for the entire day. This is a tiny company (appears 30 or so people?) that just scored a 2B valuation, produced easily the most performant 7B model and a 7B*8 MOE model that performs at the level of a 70B requiring the inference power of a 14B. I feel this could be a potential bigger threat to OpenAI than Google or Anthropic. I gather with the huge recent investment they'll be able to a) scale out to a reasonable traffic load in the near future and b) attract the best and brightest researchers put off w/ various chest puffing and drama that has been front and center in this industry.

huevosabio 3y ago
Agreed. By far the most impressive company out of the current AI wave. They managed to put out the gold standard for 7B models in like 6 months, and are quickly moving up the scale. I mocked the funding round back in March as being signs of hype, ($300m for a team of 3 with just an idea?) but clearly I didn't know the details. Really remarkable execution. They may well be on their way to eat all use cases that don't need gpt-4 performance and hopefully soon tackle the big leagues as well. Exciting times!
- appplication 3y ago
  In some ways I see this as a sign that LLMs possibly aren’t that hard to iterate on now that the cat is out of the bag. A small team can push the state of the art in just 6 months? It feels like this is more a circumstance of the state of LLMs presently and less a statement on Mistral. I wouldn’t it be surprised if we see 10+ other new $1B+ companies spring up because of how much room for improvement there is in the space.
  - a_wild_dandan 3y ago
    Well said. OpenAI's biggest contribution wasn't serving GPT-4, it was proving that big LLMs are AGIs. Rushing to gold is easy; discovering it ain't. Since we've found a correct path, low-hanging fruit is plentiful. Throw more compute at it. Work smarter not harder (optimize). Graft on simple ideas for big gainz (e.g. RAG, MoE, chain-of-thought, etc). We moved from "GPT Who?" to...all of this, in a year. I now run an AGI at a bookish 13-year-old's level on my MacBook. The trend of hardware/software advances are astonishing. And this is just the beginning.
    - fennecbutt 3y ago
      AGI, thanks 4 lmao. Either that or you're trolling.
      - michaelbrave 3y ago
        I figure AGI will need a lot of parts to it, the same way our brain does, but having the language part figured out is I would guess is about 1/4 of what we need to solve to get there. So it's not AGI, but I would say it's a big chunk of what we need to get to AGI.
        berniedurfee 3y ago
        I have to disagree. I think language will be emergent from AGI, not the other way round. LLMs are neat, but likely wont have a place in an AGI.
ru552 3y ago
Mistral is in France though and the EU is throwing cold water on "AI" at the moment. I think this hurts them in the long run.
- _fizz_buzz_ 3y ago
  The EU makes in it’s proposed law a lot of exceptions for open models. It might be a big selling point for mistral (or Facebook) but maybe tricky to deal with for closed models like OpenAI.
- pavlov 3y ago
  Or it could help them be more globally competitive because they’ll have built their product to accommodate the EU rules and will be ready if/when other countries introduce something similar. Maybe in 1970 someone was arguing that Volvo is handicapped because Swedish safety regulations have become so stringent that their cars can’t compete on price elsewhere. Turns out it was exactly Volvo’s strength to be ahead of the curve on regulation.
  - ru552 3y ago
    It's possible, but the AI space right now is all about the vibes and even the French President isn't feeling it[0]. The job market for AI/ML people is white hot right now and this is something Mistral has to try to explain around in an interview. I hope they do well despite it. [0]https://www.ft.com/content/9339d104-7b0c-42b8-9316-72226dd4e4c0 https://www.ft.com/content/9339d104-7b0c-42b8-9316-72226dd4e...
  - dmix 3y ago
    Regulations are accumulative though, not just relative to your era. So a critique in 2023 would be more relevant than one in 1970 as the pure scale of it has increased dramatically. I've seen a few graphs showing staffing counts and budgets for US regulatory agencies have grown exponentially since the 1970s (adjusted for inflation), while it was relatively flat before then in the 60s. Economic regulation growth in the US has grown steadily while social regulations (workplace, climate, healthcare, transportation etc) grew dramatically. Adding in TSA / Homeland security and it's a hockey stick. https://regulatorystudies.columbian.gwu.edu/sites/g/files/zaxdzs4751/files/downloads/RegulatorsBudget/GW%20Reg%20Studies%20-%20FY2017%20Regulators%20Budget%20-%20SDudley%20and%20MWarren.pdf https://regulatorystudies.columbian.gwu.edu/sites/g/files/za... I'd be curious to see similar charts for the EU.
dmix 3y ago
> I'm surprised this isn't firmly attached to the top of HN right now for the entire day. It's kind of hard to tell what it is from both the blog post and their homepage. So only people really familiar with AI will grasp the relevance. But your comment certainly helps.
0xDEF 3y ago
French engineering and computer science education really emphasizes math and theory. That is an advantage in AI.
dang 3y ago
It's because there have been quite a few big threads already: Mixtral of experts - https://news.ycombinator.com/item?id=38598559 https://news.ycombinator.com/item?id=38598559 - Dec 2023 (272 comments) Mistral-8x7B-Chat - https://news.ycombinator.com/item?id=38594578 https://news.ycombinator.com/item?id=38594578 - Dec 2023 (69 comments) Mistral AI Valued at $2B - https://news.ycombinator.com/item?id=38593616 https://news.ycombinator.com/item?id=38593616 - Dec 2023 (221 comments) Mistral's mixtral-8x7B-32kseqlen on Vercel - https://news.ycombinator.com/item?id=38584179 https://news.ycombinator.com/item?id=38584179 - Dec 2023 (30 comments) French AI startup Mistral secures €2B valuation - https://news.ycombinator.com/item?id=38580758 https://news.ycombinator.com/item?id=38580758 - Dec 2023 (76 comments) Mistral "Mixtral" 8x7B 32k model [magnet] - https://news.ycombinator.com/item?id=38570537 https://news.ycombinator.com/item?id=38570537 - Dec 2023 (236 comments) I know these are not all the same exact story but the discussions end up being more or less generically the same, so we can call them all (or most of them) quasidupes.
- brandall10 3y ago
  Yes, I'm aware there's been a ton of recent chatter. Most of these are dupes though so it's really just Mixtral, their funding, and now their commercial offering. My argument is opening up the commercial offering is perhaps the biggest story yet simply because of all excitement this company has been able to generate.

huevosabio 3y ago

Agreed. By far the most impressive company out of the current AI wave. They managed to put out the gold standard for 7B models in like 6 months, and are quickly moving up the scale. I mocked the funding round back in March as being signs of hype, ($300m for a team of 3 with just an idea?) but clearly I didn't know the details. Really remarkable execution. They may well be on their way to eat all use cases that don't need gpt-4 performance and hopefully soon tackle the big leagues as well. Exciting times!

appplication 3y ago
In some ways I see this as a sign that LLMs possibly aren’t that hard to iterate on now that the cat is out of the bag. A small team can push the state of the art in just 6 months? It feels like this is more a circumstance of the state of LLMs presently and less a statement on Mistral. I wouldn’t it be surprised if we see 10+ other new $1B+ companies spring up because of how much room for improvement there is in the space.
- a_wild_dandan 3y ago
  Well said. OpenAI's biggest contribution wasn't serving GPT-4, it was proving that big LLMs are AGIs. Rushing to gold is easy; discovering it ain't. Since we've found a correct path, low-hanging fruit is plentiful. Throw more compute at it. Work smarter not harder (optimize). Graft on simple ideas for big gainz (e.g. RAG, MoE, chain-of-thought, etc). We moved from "GPT Who?" to...all of this, in a year. I now run an AGI at a bookish 13-year-old's level on my MacBook. The trend of hardware/software advances are astonishing. And this is just the beginning.
  - fennecbutt 3y ago
    AGI, thanks 4 lmao. Either that or you're trolling.
    - michaelbrave 3y ago
      I figure AGI will need a lot of parts to it, the same way our brain does, but having the language part figured out is I would guess is about 1/4 of what we need to solve to get there. So it's not AGI, but I would say it's a big chunk of what we need to get to AGI.
      - berniedurfee 3y ago
        I have to disagree. I think language will be emergent from AGI, not the other way round. LLMs are neat, but likely wont have a place in an AGI.

ru552 3y ago

Mistral is in France though and the EU is throwing cold water on "AI" at the moment. I think this hurts them in the long run.

_fizz_buzz_ 3y ago
The EU makes in it’s proposed law a lot of exceptions for open models. It might be a big selling point for mistral (or Facebook) but maybe tricky to deal with for closed models like OpenAI.
pavlov 3y ago
Or it could help them be more globally competitive because they’ll have built their product to accommodate the EU rules and will be ready if/when other countries introduce something similar. Maybe in 1970 someone was arguing that Volvo is handicapped because Swedish safety regulations have become so stringent that their cars can’t compete on price elsewhere. Turns out it was exactly Volvo’s strength to be ahead of the curve on regulation.
- ru552 3y ago
  It's possible, but the AI space right now is all about the vibes and even the French President isn't feeling it[0]. The job market for AI/ML people is white hot right now and this is something Mistral has to try to explain around in an interview. I hope they do well despite it. [0]https://www.ft.com/content/9339d104-7b0c-42b8-9316-72226dd4e4c0 https://www.ft.com/content/9339d104-7b0c-42b8-9316-72226dd4e...
- dmix 3y ago
  Regulations are accumulative though, not just relative to your era. So a critique in 2023 would be more relevant than one in 1970 as the pure scale of it has increased dramatically. I've seen a few graphs showing staffing counts and budgets for US regulatory agencies have grown exponentially since the 1970s (adjusted for inflation), while it was relatively flat before then in the 60s. Economic regulation growth in the US has grown steadily while social regulations (workplace, climate, healthcare, transportation etc) grew dramatically. Adding in TSA / Homeland security and it's a hockey stick. https://regulatorystudies.columbian.gwu.edu/sites/g/files/zaxdzs4751/files/downloads/RegulatorsBudget/GW%20Reg%20Studies%20-%20FY2017%20Regulators%20Budget%20-%20SDudley%20and%20MWarren.pdf https://regulatorystudies.columbian.gwu.edu/sites/g/files/za... I'd be curious to see similar charts for the EU.

dmix 3y ago

> I'm surprised this isn't firmly attached to the top of HN right now for the entire day. It's kind of hard to tell what it is from both the blog post and their homepage. So only people really familiar with AI will grasp the relevance. But your comment certainly helps.

0xDEF 3y ago

French engineering and computer science education really emphasizes math and theory. That is an advantage in AI.

dang 3y ago

It's because there have been quite a few big threads already: Mixtral of experts - https://news.ycombinator.com/item?id=38598559 https://news.ycombinator.com/item?id=38598559 - Dec 2023 (272 comments) Mistral-8x7B-Chat - https://news.ycombinator.com/item?id=38594578 https://news.ycombinator.com/item?id=38594578 - Dec 2023 (69 comments) Mistral AI Valued at $2B - https://news.ycombinator.com/item?id=38593616 https://news.ycombinator.com/item?id=38593616 - Dec 2023 (221 comments) Mistral's mixtral-8x7B-32kseqlen on Vercel - https://news.ycombinator.com/item?id=38584179 https://news.ycombinator.com/item?id=38584179 - Dec 2023 (30 comments) French AI startup Mistral secures €2B valuation - https://news.ycombinator.com/item?id=38580758 https://news.ycombinator.com/item?id=38580758 - Dec 2023 (76 comments) Mistral "Mixtral" 8x7B 32k model [magnet] - https://news.ycombinator.com/item?id=38570537 https://news.ycombinator.com/item?id=38570537 - Dec 2023 (236 comments) I know these are not all the same exact story but the discussions end up being more or less generically the same, so we can call them all (or most of them) quasidupes.

brandall10 3y ago
Yes, I'm aware there's been a ton of recent chatter. Most of these are dupes though so it's really just Mixtral, their funding, and now their commercial offering. My argument is opening up the commercial offering is perhaps the biggest story yet simply because of all excitement this company has been able to generate.