5 ms·

It feels like there’s a big gap in this discussion. The focus is almost entirely on GPU and hardware investment, which is undeniably driving a lot of the curren

It feels like there’s a big gap in this discussion. The focus is almost entirely on GPU and hardware investment, which is undeniably driving a lot of the current AI boom. But what’s missing is any mention of the software side and the significant VC investment going into AI-driven platforms, tools, and applications. This story could be more accurately titled ‘The GPU Investment Boom’ given how heavily it leans into the hardware conversation. Software investment deserves equal attention

pfisherman 2y ago
Also they talk a lot about data centers and cloud compute, but do not mention chips for on decide inference. Given where mobile sits in the hierarchy of interfaces, that is where I would be placing my bets if I were a VC.
- deleted 2y ago
  [deleted]
deleted 2y ago
[deleted]
_delirium 2y ago
Is there a good estimate of the split? My impression from AI startups whose operations I know something about is that a majority of their VC raise is currently going back into paying for hardware directly or indirectly, even though they aren’t hardware startups per se, but I don’t have any solid numbers.
- bbor 2y ago
  I think your analysis is spot on, based on my expertise of "reading too much hacker news since every day since March 2023". This Bain report[1] barely mentions "Independent Software Vendors" and when they do it's clearly an afterthought, and the only software focused takes I can find are[2] extremely speculative, e.g. "For every dollar invested in hardware, we expect 8-to-20 times the amount to be spent on software... While the initial wave of AI investment lays the foundational infrastructure, the next wave is clearly set to capitalise on the burgeoning AI software market." I'm hoping someone more knowledgeable about capital markets can fill us in here, I'd be curious to see some hard numbers still! Maybe this is what a Bloomberg terminal does...? Regardless, I think this makes a lot of sense; there's no clear scientific consensus on the path forward for these models other than "keep going?", so building out preparatory infrastructure is seen as the clear, safe move. As the common refrain goes: "in a gold rush, sell shovels!" As a big believer in the upcoming cognitive era of software and society, I would only add a short bit onto the end of that saying: "...until the hydraulic mining cannons[3] come online." [1] https://www.bain.com/insights/ais-trillion-dollar-opportunity-tech-report-2024/ https://www.bain.com/insights/ais-trillion-dollar-opportunit... [2] https://www.privatebankerinternational.com/comment/is-ai-software-the-next-big-investment-opportunity/?cf-view https://www.privatebankerinternational.com/comment/is-ai-sof... [3] https://en.wikipedia.org/wiki/Hydraulic_mining https://en.wikipedia.org/wiki/Hydraulic_mining
  - 0xDEAFBEAD 2y ago
    >As the common refrain goes: "in a gold rush, sell shovels!" What happens when everyone constructs a shovel factory, the shovels become dirt cheap, but there are no buyers?
aurareturn 2y ago
I think GPUs and datacenter is to AI is what fiber was to the dotcom boom. A lot of LLM based software is uneconomical because we don't have enough compute and electricity for what they're trying to do.
- bee_rider 2y ago
  The actual physical fiber was useful after the companies popped though. GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years. The ecosystem for GPGPU software and the ability to design and manufacture new GPUs might be like fiber. But that is different because it doesn’t become a useful thing at rest, it only works while Nvidia (or some successor) is still running. I do think that ecosystem will stick around. Whatever the next thing after AI is, I bet Nvidia has a good enough stack at this point to pivot to it. They are the vendor for these high-throughput devices: CPU vendors will never keep up with their ability to just go wider, and coders are good enough nowadays to not need the crutch of lower latency that CPUs provide (well actually we just call frameworks written by cleverer people, but borrowing smarts is a form of cleverness). But we do need somebody to keep releasing new versions of CUDA.
  - aurareturn 2y ago
    But computer chips have always had limited usefulness because newer chips are simply faster and more efficient. The datacenter build outs and increase in electricity capacity will always be useful.
  - Wheatman 2y ago
    >GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years. Not really much for gaming, especially here in third world cou tries, OLD or Abandoned Gpu's are basically all that is avalaible for use, from anything from gaming to even video editing. Considering how many new great games are being made(and with the news of nvidia drivers possibly becoming easily avalaible on linux.), and with tech becoming more avalaible and usefull on these places, i expect there to be a somewhat considerable increase in demand for GPU in say here in africa or southeast asia. It probably wont change the world or us economy, but it would pribably make me quite happy if the bubble were to burst, even as a supporter of AI in research and Cancer detection.
_fat_santa 2y ago
Regarding software, one thing I've noticed looking at YC and Product Hunt these days is that pretty much all the software being hyped now is "AI Powered...something". I find it quite annoying because for every company that is doing something where AI would actually be useful, there are 10 that are shoving it into their existing app in some capacity to make their software "AI Powered". A perfect example of this is my company recently evaluated Zenhub. Their sales team was very eager to point out that their app was using AI though when we actually looked, all it did was generate story descriptions from a prompt, the most basic of AI integrations. AI is very useful but my god not everything needs to have it baked in.
- n_ary 2y ago
  I am eagerly waiting for my Dishwasher, Washing machine, Smart TV, electric stove and everything household electronics becomes AI powered. At least my phillips Hue does not yet have an AI and I am very happy about it. Honorable mention to the IoT, SmartHome, ConnectedHome and other old hypes which we all forgot. May be, SmartHome will become an actual reality, if we can somehow get the LLMs to make autonomous decisions to keep a house maintained and comfy.
  - Wheatman 2y ago
    Man, you would the new samaung AI fridge[1]. Then there is the AI bloatware [2] in your operating system that pinky promises it wont spy one you, even as it is becoming harder and harder to turn off. [1]https://www.theverge.com/2023/12/27/24016939/samsung-2024-ai-family-hub-smart-fridge-features https://www.theverge.com/2023/12/27/24016939/samsung-2024-ai.... [2]Mostly rumors about copilot, please don't take this as gospel.

pfisherman 2y ago

Also they talk a lot about data centers and cloud compute, but do not mention chips for on decide inference. Given where mobile sits in the hierarchy of interfaces, that is where I would be placing my bets if I were a VC.

deleted 2y ago
[deleted]

deleted 2y ago

[deleted]

_delirium 2y ago

Is there a good estimate of the split? My impression from AI startups whose operations I know something about is that a majority of their VC raise is currently going back into paying for hardware directly or indirectly, even though they aren’t hardware startups per se, but I don’t have any solid numbers.

bbor 2y ago
I think your analysis is spot on, based on my expertise of "reading too much hacker news since every day since March 2023". This Bain report[1] barely mentions "Independent Software Vendors" and when they do it's clearly an afterthought, and the only software focused takes I can find are[2] extremely speculative, e.g. "For every dollar invested in hardware, we expect 8-to-20 times the amount to be spent on software... While the initial wave of AI investment lays the foundational infrastructure, the next wave is clearly set to capitalise on the burgeoning AI software market." I'm hoping someone more knowledgeable about capital markets can fill us in here, I'd be curious to see some hard numbers still! Maybe this is what a Bloomberg terminal does...? Regardless, I think this makes a lot of sense; there's no clear scientific consensus on the path forward for these models other than "keep going?", so building out preparatory infrastructure is seen as the clear, safe move. As the common refrain goes: "in a gold rush, sell shovels!" As a big believer in the upcoming cognitive era of software and society, I would only add a short bit onto the end of that saying: "...until the hydraulic mining cannons[3] come online." [1] https://www.bain.com/insights/ais-trillion-dollar-opportunity-tech-report-2024/ https://www.bain.com/insights/ais-trillion-dollar-opportunit... [2] https://www.privatebankerinternational.com/comment/is-ai-software-the-next-big-investment-opportunity/?cf-view https://www.privatebankerinternational.com/comment/is-ai-sof... [3] https://en.wikipedia.org/wiki/Hydraulic_mining https://en.wikipedia.org/wiki/Hydraulic_mining
- 0xDEAFBEAD 2y ago
  >As the common refrain goes: "in a gold rush, sell shovels!" What happens when everyone constructs a shovel factory, the shovels become dirt cheap, but there are no buyers?

aurareturn 2y ago

I think GPUs and datacenter is to AI is what fiber was to the dotcom boom. A lot of LLM based software is uneconomical because we don't have enough compute and electricity for what they're trying to do.

bee_rider 2y ago
The actual physical fiber was useful after the companies popped though. GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years. The ecosystem for GPGPU software and the ability to design and manufacture new GPUs might be like fiber. But that is different because it doesn’t become a useful thing at rest, it only works while Nvidia (or some successor) is still running. I do think that ecosystem will stick around. Whatever the next thing after AI is, I bet Nvidia has a good enough stack at this point to pivot to it. They are the vendor for these high-throughput devices: CPU vendors will never keep up with their ability to just go wider, and coders are good enough nowadays to not need the crutch of lower latency that CPUs provide (well actually we just call frameworks written by cleverer people, but borrowing smarts is a form of cleverness). But we do need somebody to keep releasing new versions of CUDA.
- aurareturn 2y ago
  But computer chips have always had limited usefulness because newer chips are simply faster and more efficient. The datacenter build outs and increase in electricity capacity will always be useful.
- Wheatman 2y ago
  >GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years. Not really much for gaming, especially here in third world cou tries, OLD or Abandoned Gpu's are basically all that is avalaible for use, from anything from gaming to even video editing. Considering how many new great games are being made(and with the news of nvidia drivers possibly becoming easily avalaible on linux.), and with tech becoming more avalaible and usefull on these places, i expect there to be a somewhat considerable increase in demand for GPU in say here in africa or southeast asia. It probably wont change the world or us economy, but it would pribably make me quite happy if the bubble were to burst, even as a supporter of AI in research and Cancer detection.

_fat_santa 2y ago

Regarding software, one thing I've noticed looking at YC and Product Hunt these days is that pretty much all the software being hyped now is "AI Powered...something". I find it quite annoying because for every company that is doing something where AI would actually be useful, there are 10 that are shoving it into their existing app in some capacity to make their software "AI Powered". A perfect example of this is my company recently evaluated Zenhub. Their sales team was very eager to point out that their app was using AI though when we actually looked, all it did was generate story descriptions from a prompt, the most basic of AI integrations. AI is very useful but my god not everything needs to have it baked in.

n_ary 2y ago
I am eagerly waiting for my Dishwasher, Washing machine, Smart TV, electric stove and everything household electronics becomes AI powered. At least my phillips Hue does not yet have an AI and I am very happy about it. Honorable mention to the IoT, SmartHome, ConnectedHome and other old hypes which we all forgot. May be, SmartHome will become an actual reality, if we can somehow get the LLMs to make autonomous decisions to keep a house maintained and comfy.
- Wheatman 2y ago
  Man, you would the new samaung AI fridge[1]. Then there is the AI bloatware [2] in your operating system that pinky promises it wont spy one you, even as it is becoming harder and harder to turn off. [1]https://www.theverge.com/2023/12/27/24016939/samsung-2024-ai-family-hub-smart-fridge-features https://www.theverge.com/2023/12/27/24016939/samsung-2024-ai.... [2]Mostly rumors about copilot, please don't take this as gospel.