13 ms·

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that

by dvt 2y ago

Infra has always been a tarpit idea. Google didn't start out as an "infra" company, and neither did Amazon or Facebook. In fact, the few "infra companies" that did start back then (companies like Godaddy) are minuscule compared to the aforementioned. VC pouring money in LLM infra is legitimately crazy to me. It's clear as day that there will be winners of this AI cycle, but, as always, they will be companies that provide actual, real, tangible value. Making shovels works for huge companies like Nvidia or Intel, but it won't work for you. It's sad to see so much capital funneled in frameworks upon frameworks upon frameworks instead of fresh new ideas that could revolutionize the way we interact with our devices. I know it's a bit of a meme, but I'd rather see more Rabbit R1 and less LangChain. Even OpenAI doesn't really have a product. Just throwing data at a bunch of video cards isn't value-generating in itself. We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better.

rapsey 2y ago
> Even OpenAI doesn't really have a product. They are making a ton of money off subscriptions.
- nextworddev 2y ago
  Yes. Author here. OpenAI is at near 4bn ARR and growing (api plus SaaS)
  - whiplash451 2y ago
    Source?
    - matwood 2y ago
      https://www.theinformation.com/articles/openais-annualized-revenue-doubles-to-3-4-billion-since-late-2023 https://www.theinformation.com/articles/openais-annualized-r...
- soneca 2y ago
  Yep. Obviously they have a product. Millions and millions of people that are not software engineers or any very tech-savvy persona use their product. And a lot of those people pay for it. It’s just silly to say they don’t really have a product. Specially given the rest of GP comment, OpenAI seems to be the Google, Facebook of the industry and will be the infrastructure company of it (already is kind of)
- criddell 2y ago
  I'm skeptical that $20 / month is enough to run OpenAI and be profitable. I would bet the real number is an order of magnitude higher.
  - rapsey 2y ago
    I did not say profitable
    - deleted 2y ago
      [deleted]
  - phillipcarter 2y ago
    OpenAI is in growth mode, not profitability mode. Their ARR is over 3B now, and doesn't show signs of stagnating. At some point they will need to build and offer more products, but they've got as good a shot at being the next big tech company as anyone else.
siquick 2y ago
> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.
- dvt 2y ago
  In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product.
  - siquick 2y ago
    Uploading a a photo/file to a server does not a product make yet you referenced Dropbox and Instagram and that’s what they started out as. The UX of the AI applications is the moat and the infrastructure providers behind those applications is pretty much always OpenAI and Anthropic at the moment because running your own open source LLMs (which are inferior out of the box) at scale is not cheap or easy to do it right - same reason most companies use cloud. Agree that once you hit super scale then you can run your own infrastructure but there are thousand of companies who won’t get that far and still need an LLM.
    - iknownthing 2y ago
      Agreed. The LLMs have effectively been democratized. As much as people deride "LLM wrappers", the quality of the wrapper and how creatively they use the LLM api is the differentiator.
  - fragmede 2y ago
    That's one opinion. the number off people who get value out of ChatGPT is known only to them, but anecdotally it's pretty high. a lot of people I know are actively using it on a daily basis and paying them $20/month.
rapsey 2y ago
> VC pouring money in LLM infra is legitimately crazy to me. VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.
- ozim 2y ago
  I think they don't even trail S&P 500 - with all the dirty tricks up their sleeves they roll losses over and over. Bottom will fall off at some point when hype train stops rolling, but so far it was good years as we have hype after hype. I am waiting what will be next one after AI, because quantum computing feels like too hard to become a hype, the same with space ventures, there is some upward trend going on there but still space is too hard.
  - rapsey 2y ago
    AI is a long term trend, next is more products on top of AI. Just like the internet. Biotech and space will both be trends but slower and less bubbly because the cost to play is high, though the returns are possibly huge in both.
    - ToucanLoucan 2y ago
      > AI is a long term trend, next is more products on top of AI. Just like the internet. This is legitimately just the same damn hype train the tech sector is constantly attempting to create. Now AI is the next internet. Before that it was the metaverse. Before that it was NFTs. Before that it was cryptocurrency. Before that it was quantum. Before that it was VR. Before that it was AR. None of those were the revolutions postured by techno-fetishistic CEOs. Most stick around in some capacity, like VR and AR, and the argument can be made that those have a future. Blockchains certainly have a future as it's a highly useful technology, even if the financial vehicle made by it is utterly useless. The metaverse is Dead on Arrival because nobody ever wanted it in the first damn place, apart from the speculators betting money on it.
nextworddev 2y ago
Author here. I think the tarpit extends to most chatgpt wrappers as well, which is why I called out pivoting prematurely to application layer is a futile exercise.
- iknownthing 2y ago
  So AI infrastructure startups are tarpits and so are the wrappers? Is it just tarpits all the way down?
  - nextworddev 2y ago
    It's an area where if you don't have a clear edge, it could be a real tarpit.
tryauuum 2y ago
are there even any VCs pouring money in LLM infra? I would assume VCs aren't interested in projects which won't give them a tenfold return. And with infrastructure there are some many existing competitors (like AWS) so that such returns are never expected
- __MatrixMan__ 2y ago
  I get the feeling that there are. MLOps became LLMOps before most people knew it as any more than a buzzword and well we can't just call it ops because there's clearly something different here. AI is still too opaque to reliably know beforehand if an idea will pan out, so you just gotta try it. Plus it's easy, once you start imagining how the magic of AI is gonna make you rich, to ignore the problems with your idea and assume that the AI will handle them too. So you've got all these hyped up fools trying to make stuff that lacks merit. How do you capitalize on that? You don't invest in the stuff that's doomed to fail, you create a slot: > Insert coin, insert half baked idea, receive AI app That way you get to keep the coins even when apps don't turn out. Plus, you're collecting the institutional knowledge necessary to pounce on something that comes along which is actually worth investing in. Or at least that's the vibe I get when our meetings feature an AI-excited VC (which isn't common, but it happens).
belter 2y ago
> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. None of those three make my life better or easier: Dropbox -> We drop your data directly on S3 but give you worst security... Slack -> Being able to navigate our messy interface is an IQ test in itself... Instagram -> Only makes your life better if you have bikini posts to share....
- raincole 2y ago
  > None of those three make my life better or easier: > Dropbox -> We drop your data directly on S3 but we give you worst security... You're top 1% tech-savvy then. 99% people don't know what S3 is and won't deem Dropbox as a worse S3. (insert rsync joke)
- hparadiz 2y ago
  I remember what it was like before Dropbox and Slack and I can tell you both have saved me hundreds of hours directly. Can't say that for IG.
  - belter 2y ago
    > I remember what it was like before Dropbox And despite all that, I get mysteriously being pushed by partner companies I work with, this product named Box. With tactics similar to the push to use Teams. It's like being volunteered for karaoke night by your Japanese boss...
- MOARDONGZPLZ 2y ago
  All three of these products have made my life substantially better.
  - belter 2y ago
    Next you are going to recommend me Windows and Teams? :-)
- oefrha 2y ago
  Didn’t expect curlftpfs+svn to strike again in 2024.
raincole 2y ago
> Even OpenAI doesn't really have a product. ...What? Like, what? I feel like stepping into another timeline reading this.
williamcotton 2y ago
Even OpenAI doesn't really have a product. This becomes very obvious when you use Claude Projects with Artifacts! ChatGPT depends heavily on one’s ability to copy and paste… and even though it is an improvement, Claude Projects still make managing a set of documents tedious compared to your standard code editor. Third-party tools like Cursor are an improvement but will be prohibitively expensive compared to companies that create and manage their own LLMs. I expect to see a native document editing/code editing software system directly from one of the LLMs-as-a-service companies at some point.
tcgv 2y ago
> Even OpenAI doesn't really have a product. Just throwing data at a bunch of video cards isn't value-generating in itself. We need (...) something people love that makes their life easier or better. I agree. I'm of the hypothesis that, when it comes to AI, a lot of product teams are pursuing overly ambitious and sophisticated features instead of targeting easy wins that are in plain sight [1]. [1] https://thomasvilhena.com/2024/06/easy-wins-for-generative-ai https://thomasvilhena.com/2024/06/easy-wins-for-generative-a...

by dvt 2y ago

rapsey 2y ago

> Even OpenAI doesn't really have a product. They are making a ton of money off subscriptions.

nextworddev 2y ago
Yes. Author here. OpenAI is at near 4bn ARR and growing (api plus SaaS)
- whiplash451 2y ago
  Source?
  - matwood 2y ago
    https://www.theinformation.com/articles/openais-annualized-revenue-doubles-to-3-4-billion-since-late-2023 https://www.theinformation.com/articles/openais-annualized-r...
soneca 2y ago
Yep. Obviously they have a product. Millions and millions of people that are not software engineers or any very tech-savvy persona use their product. And a lot of those people pay for it. It’s just silly to say they don’t really have a product. Specially given the rest of GP comment, OpenAI seems to be the Google, Facebook of the industry and will be the infrastructure company of it (already is kind of)
criddell 2y ago
I'm skeptical that $20 / month is enough to run OpenAI and be profitable. I would bet the real number is an order of magnitude higher.
- rapsey 2y ago
  I did not say profitable
  - deleted 2y ago
    [deleted]
- phillipcarter 2y ago
  OpenAI is in growth mode, not profitability mode. Their ARR is over 3B now, and doesn't show signs of stagnating. At some point they will need to build and offer more products, but they've got as good a shot at being the next big tech company as anyone else.

siquick 2y ago

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. People seem to not mind ChatGPT or Claude and safe to say that a very large majority of AI products are using one of the APIs of those companies.

dvt 2y ago
In my humble opinion, a chat interface (API or not) does not a product make. Not to mention that Llama is free and competitive with both (same with Mistral, heck the 7B model works great on my RTX 3080). If you started a company that blew up because you made a badass product (and let's say you used ChatGPT under the hood), you would just eventually train and deploy your own model because an LLM is not a product.
- siquick 2y ago
  Uploading a a photo/file to a server does not a product make yet you referenced Dropbox and Instagram and that’s what they started out as. The UX of the AI applications is the moat and the infrastructure providers behind those applications is pretty much always OpenAI and Anthropic at the moment because running your own open source LLMs (which are inferior out of the box) at scale is not cheap or easy to do it right - same reason most companies use cloud. Agree that once you hit super scale then you can run your own infrastructure but there are thousand of companies who won’t get that far and still need an LLM.
  - iknownthing 2y ago
    Agreed. The LLMs have effectively been democratized. As much as people deride "LLM wrappers", the quality of the wrapper and how creatively they use the LLM api is the differentiator.
- fragmede 2y ago
  That's one opinion. the number off people who get value out of ChatGPT is known only to them, but anecdotally it's pretty high. a lot of people I know are actively using it on a daily basis and paying them $20/month.

rapsey 2y ago

> VC pouring money in LLM infra is legitimately crazy to me. VC business model is throwing money at the wall and seeing what sticks. They love congratulating themselves on how smart they are but at the end of the day their overall returns trail S&P 500. They are salespeople and their job is to sell themselves to private capital on how smart and connected they are.

ozim 2y ago
I think they don't even trail S&P 500 - with all the dirty tricks up their sleeves they roll losses over and over. Bottom will fall off at some point when hype train stops rolling, but so far it was good years as we have hype after hype. I am waiting what will be next one after AI, because quantum computing feels like too hard to become a hype, the same with space ventures, there is some upward trend going on there but still space is too hard.
- rapsey 2y ago
  AI is a long term trend, next is more products on top of AI. Just like the internet. Biotech and space will both be trends but slower and less bubbly because the cost to play is high, though the returns are possibly huge in both.
  - ToucanLoucan 2y ago
    > AI is a long term trend, next is more products on top of AI. Just like the internet. This is legitimately just the same damn hype train the tech sector is constantly attempting to create. Now AI is the next internet. Before that it was the metaverse. Before that it was NFTs. Before that it was cryptocurrency. Before that it was quantum. Before that it was VR. Before that it was AR. None of those were the revolutions postured by techno-fetishistic CEOs. Most stick around in some capacity, like VR and AR, and the argument can be made that those have a future. Blockchains certainly have a future as it's a highly useful technology, even if the financial vehicle made by it is utterly useless. The metaverse is Dead on Arrival because nobody ever wanted it in the first damn place, apart from the speculators betting money on it.

nextworddev 2y ago

Author here. I think the tarpit extends to most chatgpt wrappers as well, which is why I called out pivoting prematurely to application layer is a futile exercise.

iknownthing 2y ago
So AI infrastructure startups are tarpits and so are the wrappers? Is it just tarpits all the way down?
- nextworddev 2y ago
  It's an area where if you don't have a clear edge, it could be a real tarpit.

tryauuum 2y ago

are there even any VCs pouring money in LLM infra? I would assume VCs aren't interested in projects which won't give them a tenfold return. And with infrastructure there are some many existing competitors (like AWS) so that such returns are never expected

__MatrixMan__ 2y ago
I get the feeling that there are. MLOps became LLMOps before most people knew it as any more than a buzzword and well we can't just call it ops because there's clearly something different here. AI is still too opaque to reliably know beforehand if an idea will pan out, so you just gotta try it. Plus it's easy, once you start imagining how the magic of AI is gonna make you rich, to ignore the problems with your idea and assume that the AI will handle them too. So you've got all these hyped up fools trying to make stuff that lacks merit. How do you capitalize on that? You don't invest in the stuff that's doomed to fail, you create a slot: > Insert coin, insert half baked idea, receive AI app That way you get to keep the coins even when apps don't turn out. Plus, you're collecting the institutional knowledge necessary to pounce on something that comes along which is actually worth investing in. Or at least that's the vibe I get when our meetings feature an AI-excited VC (which isn't common, but it happens).

belter 2y ago

> We need a Dropbox or a Slack or an Instagram: something people love that makes their life easier or better. None of those three make my life better or easier: Dropbox -> We drop your data directly on S3 but give you worst security... Slack -> Being able to navigate our messy interface is an IQ test in itself... Instagram -> Only makes your life better if you have bikini posts to share....

raincole 2y ago
> None of those three make my life better or easier: > Dropbox -> We drop your data directly on S3 but we give you worst security... You're top 1% tech-savvy then. 99% people don't know what S3 is and won't deem Dropbox as a worse S3. (insert rsync joke)
hparadiz 2y ago
I remember what it was like before Dropbox and Slack and I can tell you both have saved me hundreds of hours directly. Can't say that for IG.
- belter 2y ago
  > I remember what it was like before Dropbox And despite all that, I get mysteriously being pushed by partner companies I work with, this product named Box. With tactics similar to the push to use Teams. It's like being volunteered for karaoke night by your Japanese boss...
MOARDONGZPLZ 2y ago
All three of these products have made my life substantially better.
- belter 2y ago
  Next you are going to recommend me Windows and Teams? :-)
oefrha 2y ago
Didn’t expect curlftpfs+svn to strike again in 2024.

raincole 2y ago

> Even OpenAI doesn't really have a product. ...What? Like, what? I feel like stepping into another timeline reading this.

williamcotton 2y ago

Even OpenAI doesn't really have a product. This becomes very obvious when you use Claude Projects with Artifacts! ChatGPT depends heavily on one’s ability to copy and paste… and even though it is an improvement, Claude Projects still make managing a set of documents tedious compared to your standard code editor. Third-party tools like Cursor are an improvement but will be prohibitively expensive compared to companies that create and manage their own LLMs. I expect to see a native document editing/code editing software system directly from one of the LLMs-as-a-service companies at some point.

tcgv 2y ago

> Even OpenAI doesn't really have a product. Just throwing data at a bunch of video cards isn't value-generating in itself. We need (...) something people love that makes their life easier or better. I agree. I'm of the hypothesis that, when it comes to AI, a lot of product teams are pursuing overly ambitious and sophisticated features instead of targeting easy wins that are in plain sight [1]. [1] https://thomasvilhena.com/2024/06/easy-wins-for-generative-ai https://thomasvilhena.com/2024/06/easy-wins-for-generative-a...