4 ms·

$100M isn't enough capital for an AI startup that's training foundation models, sadly. A ton of folks of similar stature who raised that much burnt it within t

$100M isn't enough capital for an AI startup that's training foundation models, sadly. A ton of folks of similar stature who raised that much burnt it within two years and took mediocre exits.

versteegen 2y ago
Chollet is a leading skeptic of the generality of LLMs (see arcprize.org). He surely isn't doing a startup to train another one.
NitpickLawyer 2y ago
I think we'll start to see a differentiation soon. The likes of Ilya will raise money to do whatever, including foundation models / new arch, while other startups will focus on post-training, scaling inference, domain adaptation and so on. I don't think the idea of general foundational model from scratch is a good path for startups anymore. We're already seeing specialised verticals (cursor, codeium, both at ~100-200m funding rounds) and they're both focused on specific domains, not generalist. There's probably enough "foundation" models out there to start working on post-training stuff already, no need to reinvent the wheel.
zxexz 2y ago
Interesting, I think $100M is totally enough to train a SotA "foundation model". It's all in the use case. I'd love to hear explicit arguments against this.
- hiddencost 2y ago
  There's a bunch of failed AI companies who raised been $100M and $200M with the goal of training foundation models. What they discovered is that they were rapidly out paced by the large players, and didn't have any way to generate revenue. You're right that it's enough to train one, but IMO you're wrong that it's enough to build a company around.
  - Auracle 2y ago
    I imagine Black Forest Labs (Flux) is doing alright, at least for now. I still feel like they’re missing out on some hanging fruit financially though. But yeah, you’re not going to make any money making yet another LLM unless it’s somehow special.
  - ak_111 2y ago
    can you please name names? I can't think of any (but am not an expert on the space).

$100M isn't enough capital for an AI startup that's training foundation models, sadly. A ton of folks of similar stature who raised that much burnt it within two years and took mediocre exits.

versteegen 2y ago

Chollet is a leading skeptic of the generality of LLMs (see arcprize.org). He surely isn't doing a startup to train another one.

NitpickLawyer 2y ago

I think we'll start to see a differentiation soon. The likes of Ilya will raise money to do whatever, including foundation models / new arch, while other startups will focus on post-training, scaling inference, domain adaptation and so on. I don't think the idea of general foundational model from scratch is a good path for startups anymore. We're already seeing specialised verticals (cursor, codeium, both at ~100-200m funding rounds) and they're both focused on specific domains, not generalist. There's probably enough "foundation" models out there to start working on post-training stuff already, no need to reinvent the wheel.

zxexz 2y ago

Interesting, I think $100M is totally enough to train a SotA "foundation model". It's all in the use case. I'd love to hear explicit arguments against this.

hiddencost 2y ago
There's a bunch of failed AI companies who raised been $100M and $200M with the goal of training foundation models. What they discovered is that they were rapidly out paced by the large players, and didn't have any way to generate revenue. You're right that it's enough to train one, but IMO you're wrong that it's enough to build a company around.
- Auracle 2y ago
  I imagine Black Forest Labs (Flux) is doing alright, at least for now. I still feel like they’re missing out on some hanging fruit financially though. But yeah, you’re not going to make any money making yet another LLM unless it’s somehow special.
- ak_111 2y ago
  can you please name names? I can't think of any (but am not an expert on the space).