10 ms·

Goldman Sachs says the return on investment for AI might be disappointing

tim333 2y ago
It's quite likely to be like the 1999 dot com boom and bust. Investors put a lot of money into things like Webvan which went bust in 2001 while a lot of the value ended up in the likes of Google which didn't float till 2004.
EasyMark 2y ago
When you’re selling shovels it’s easy to pump and dump and then say weeellllllll it might not be as big of a gold deposit as we thought it was.
haltingproblem 2y ago
https://archive.ph/5gq8M https://archive.ph/5gq8M
- avg_dev 2y ago
  also, you can view the original report here: https://www.goldmansachs.com/intelligence/pages/gs-research/gen-ai-too-much-spend-too-little-benefit/report.pdf https://www.goldmansachs.com/intelligence/pages/gs-research/... "generative AI" seems to be covered from pages 3-24 inclusive.
TibbityFlanders 2y ago
What a waste of humanity's resources.
- bugbuddy 2y ago
  Amen. I said the same thing many months ago but the true believers on HN who could not see beyond their own paychecks down voted me like I was advocating for satan.
- chaos_emergent 2y ago
  By that argument, anything that an AI system could do today that a human does right now is an incredibly massive waste of our resources :-)
cs702 2y ago
I agree with the folks at GS. Every non-tech corporation in the world is trying to figure out some kind of "AI strategy," but most executives and managers have no clue as to what they're doing or ought to be doing with respect to "AI." They're like patsies joining a very expensive game of poker they don't understand, and they are driven by FOMO, ready and eager to put down gobs of cash on the table, because they don't want to miss their chance of winning. They are on track to learn predictable lessons that cannot be learned in any other way.
- EasyMark 2y ago
  They should take the apple approach. There will be hundreds of options when they decide they really need an “AI strategy” it will fizzle out. It’s pretty obvious for Apple that they can use AI for some tasks and they’ll work it in practically as opposed to panic Keeping Up With the Jones’ mode. Apple is doing it the right way, since they aren’t actually in the AI business (yet), just a consumer. This is the way people need to think. When an actual use for LLMs presents itself then there will be plenty options, no need to panic.
- artemonster 2y ago
  replace "AI" with any past trendy buzzword (big data, ML, XML, XaaS) and your statement remains exactly the same, hence, carrying 0 new information. just a historical observation :)
  - Sharlin 2y ago
    Exactly. It's incredible how it's the exact same story every time, repeating every few years.
  - llm_trw 2y ago
    Replace it with the internet and it's the difference between Amazon and Sears.
    - Sharlin 2y ago
      Yes, I mean, in the 00s bubble everybody and their dog needed an internet presence without any idea of what to do with it. The difference is not that between Amazon and Sears, but that between Amazon and the thousands of contemporaneous companies with clueless execs who had no idea what to do with an internet but were told that they now HAD to have one, preferably two.
cjk2 2y ago
Every investment company out there knew this from day one. They were riding the hype and gains. Now the late and individual investors will pay for the losses while the big investors start moving cash to the next hype.
- thanksgiving 2y ago
  What does this mean for passive index funds? Is it likely that actively managed funds will have an edge over passive index funds in these scenario? How will the big investors know when to start moving money to the “next hype”?
  - cjk2 2y ago
    I think you are crazy putting money into passive index funds. As for the investors, knowing when to pull out is when they start pushing articles like this. They’re already out and want your capital to go where they are going next.
    - tananaev 2y ago
      What are you putting your money into?
      - cjk2 2y ago
        Well I’ve pretty much pulled it out of everything and bought a house (no mortgage yay) but I rode NVidia, BAE Systems with a safe position held on an actively managed ETF. I’ve got a huge pension lined up so I’m going to go on holiday a lot basically. I get a lump sum early payment of that soon so I can invest that somewhere more productive.
      - hanniabu 2y ago
        Ethereum
      - loloquwowndueo 2y ago
        Dogecoin!
sokoloff 2y ago
The return on investment on anything that’s over-bid due to hype is usually quite poor. I don’t see strong reasons to think AI will be different than tulips or South Sea investments in that regard.
- rchaud 2y ago
  "AI" is talked about like an individual company instead of like a mortgage-backed security, whose intrinsic value is made up of lots of little mortgages, some of which will pay out, and some will go into default.
blindriver 2y ago
Is this not expected? Every hype encounters a “Trough of Despair”. I think costs need to decrease dramatically before the current state of AI is worth the money that is being dumped into it. Both in terms of GPU costs and electricity because it’s insane.
- cj 2y ago
  What boggles my mind is: what happens to Nividia’s 3T market cap if GPU costs come down by 10x or 100x? Will there be 10x or 100x increase in unit sales volume needed to offset the price decrease while maintaining their 3T market cap? I really wish there were an easy way to remove Nivida from my (mostly passive ETF) portfolio. I started purchasing NVDA puts simply to protect gains on SPY holdings where I simply want to reduce my Nividia exposure.
raytopia 2y ago
Seeing that AI is being subsidized by investors I wonder how long it'll take for said investors to want a return on their investments and if AI is too expensive for anyone to actually want to use it. Because I don't think a lot of people would pay the actual cost it takes to run AI.
d_silin 2y ago
AI winter is coming! Fascinating how the bust part "boom-bust" cycle invariably comes for every promising tech. On a good side, we have finally have the first generation of AGI. Give it another ten years of improvements before we reach the next AI boom.
chaos_emergent 2y ago
I’m confused by the evidence they use: > to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do Planning and reasoning are the two greatest areas of research in AI right now, with an OOM more researchers devoted to it than there were to the first generation of generative AI architectures > In our experience, even basic summarization tasks often yield illegible and nonsensical results Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs. > even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable We’ve seen a literal 10x decrease in cost from gpt-4-32k to gpt-4o in a single year of AI development (3:1 cost blend). And that ignores that sonnet-3.5 is 50x cheaper than gpt-4-32k while getting better scores on pretty much all benchmarks? > the human brain is 10,000x more effective per unit of power in performing cognitive tasks vs. generative AI Patently false, we’re not untethered brains floating around and require shelter, food, and a ton of other energy intensive requirements to live, and an AI system can perform a task that it is designed to do easily 10-20x faster than a human could. If anything this makes me more bullish about AI systems having a positive ROI; the criticisms they have are based on extraordinarily (if not nefariously) dumb assumptions.
- dist-epoch 2y ago
  > the human brain is 10,000x more effective per unit of power Even if true, is completely irrelevant. What matters is effectiveness per dolar, not per watt. Does Goldman Sachs have an energy budget per employee, do they hire based on energy consumption? Do they prefer hiring someone living in a small house versus someone living in a mansion with a gigantic energy bill when both request the same salary?
  - chaos_emergent 2y ago
    Yeah, an incredibly dumb analysis. I did a back of the hand estimate based off of the following assumptions: gpt-4o: 10 H100s = ~7000W power requirement, 30 seconds to generate a summary for an hour-long call human: ~10.7 kWh/year energy requirement (avg per American), 20 minutes to generate the same summary The numbers come out to 60 Wh for 4o and 400 Wh for a human
sircastor 2y ago
I know that we "want" investment in AI, but I'd be perfectly happy if Wallstreet stays the heck out of this whole situation. Every time investment money gets into a topic - not company, not an industry, a topic - it becomes an absolute mess. Investment feels like a micromanager that won't let you do your work.
newsclues 2y ago
If we replace bankers with AI the returns, the returns may be better.
bparsons 2y ago
Here I am still waiting for all those amazing innovations that the blockchain will bring us.
ben_jones 2y ago
In real time we’ve seen “LLMs are the future” becomethe AI research into AGI is the future”. What’s the next movement of the goal post?
rongenre 2y ago
I'm sort of dreading the next year or so at big corps, in which engineers will hear from their management: "We've made a big investment in AI, and need you to make this work".
sneed_chucker 2y ago
Integration of LLMs with existing compute and data systems will have an excellent return on investment if done correctly. The current brain dead spitball method of shoehorning a chatbot interface on top of every single existing GUI application is not that.
deleted 2y ago
[deleted]
rsynnott 2y ago
Heresy! Kind of a little surprised that they’re coming right out and saying it at this point; I didn’t think we were at that point in the hype cycle just yet.
- disqard 2y ago
  It might be a bit of a "Prisoner's Dilemma" situation, and by being the first (among hype hawkers) to point out the Emperor's lack of clothing, they're hoping to come out ahead by throwing their competitors under the bus first. If this catches on, it'll mimic the lets-do-layoffs meme spreading virally among tech corps -- IIRC it was Elon's "look I can layoff most of the company and it'll still keep going" which started that trend. Edit: Let's not forget that Goldman Sach will definitely be betting on these outcomes (puts, etc.), so by pushing this narrative, they're definitely going to benefit.

It's quite likely to be like the 1999 dot com boom and bust. Investors put a lot of money into things like Webvan which went bust in 2001 while a lot of the value ended up in the likes of Google which didn't float till 2004.

EasyMark 2y ago

When you’re selling shovels it’s easy to pump and dump and then say weeellllllll it might not be as big of a gold deposit as we thought it was.

haltingproblem 2y ago

https://archive.ph/5gq8M https://archive.ph/5gq8M

avg_dev 2y ago
also, you can view the original report here: https://www.goldmansachs.com/intelligence/pages/gs-research/gen-ai-too-much-spend-too-little-benefit/report.pdf https://www.goldmansachs.com/intelligence/pages/gs-research/... "generative AI" seems to be covered from pages 3-24 inclusive.

TibbityFlanders 2y ago

What a waste of humanity's resources.

bugbuddy 2y ago
Amen. I said the same thing many months ago but the true believers on HN who could not see beyond their own paychecks down voted me like I was advocating for satan.
chaos_emergent 2y ago
By that argument, anything that an AI system could do today that a human does right now is an incredibly massive waste of our resources :-)

cs702 2y ago

I agree with the folks at GS. Every non-tech corporation in the world is trying to figure out some kind of "AI strategy," but most executives and managers have no clue as to what they're doing or ought to be doing with respect to "AI." They're like patsies joining a very expensive game of poker they don't understand, and they are driven by FOMO, ready and eager to put down gobs of cash on the table, because they don't want to miss their chance of winning. They are on track to learn predictable lessons that cannot be learned in any other way.

EasyMark 2y ago
They should take the apple approach. There will be hundreds of options when they decide they really need an “AI strategy” it will fizzle out. It’s pretty obvious for Apple that they can use AI for some tasks and they’ll work it in practically as opposed to panic Keeping Up With the Jones’ mode. Apple is doing it the right way, since they aren’t actually in the AI business (yet), just a consumer. This is the way people need to think. When an actual use for LLMs presents itself then there will be plenty options, no need to panic.
artemonster 2y ago
replace "AI" with any past trendy buzzword (big data, ML, XML, XaaS) and your statement remains exactly the same, hence, carrying 0 new information. just a historical observation :)
- Sharlin 2y ago
  Exactly. It's incredible how it's the exact same story every time, repeating every few years.
- llm_trw 2y ago
  Replace it with the internet and it's the difference between Amazon and Sears.
  - Sharlin 2y ago
    Yes, I mean, in the 00s bubble everybody and their dog needed an internet presence without any idea of what to do with it. The difference is not that between Amazon and Sears, but that between Amazon and the thousands of contemporaneous companies with clueless execs who had no idea what to do with an internet but were told that they now HAD to have one, preferably two.

cjk2 2y ago

Every investment company out there knew this from day one. They were riding the hype and gains. Now the late and individual investors will pay for the losses while the big investors start moving cash to the next hype.

thanksgiving 2y ago
What does this mean for passive index funds? Is it likely that actively managed funds will have an edge over passive index funds in these scenario? How will the big investors know when to start moving money to the “next hype”?
- cjk2 2y ago
  I think you are crazy putting money into passive index funds. As for the investors, knowing when to pull out is when they start pushing articles like this. They’re already out and want your capital to go where they are going next.
  - tananaev 2y ago
    What are you putting your money into?
    - cjk2 2y ago
      Well I’ve pretty much pulled it out of everything and bought a house (no mortgage yay) but I rode NVidia, BAE Systems with a safe position held on an actively managed ETF. I’ve got a huge pension lined up so I’m going to go on holiday a lot basically. I get a lump sum early payment of that soon so I can invest that somewhere more productive.
    - hanniabu 2y ago
      Ethereum
    - loloquwowndueo 2y ago
      Dogecoin!

sokoloff 2y ago

The return on investment on anything that’s over-bid due to hype is usually quite poor. I don’t see strong reasons to think AI will be different than tulips or South Sea investments in that regard.

rchaud 2y ago
"AI" is talked about like an individual company instead of like a mortgage-backed security, whose intrinsic value is made up of lots of little mortgages, some of which will pay out, and some will go into default.

blindriver 2y ago

Is this not expected? Every hype encounters a “Trough of Despair”. I think costs need to decrease dramatically before the current state of AI is worth the money that is being dumped into it. Both in terms of GPU costs and electricity because it’s insane.

cj 2y ago
What boggles my mind is: what happens to Nividia’s 3T market cap if GPU costs come down by 10x or 100x? Will there be 10x or 100x increase in unit sales volume needed to offset the price decrease while maintaining their 3T market cap? I really wish there were an easy way to remove Nivida from my (mostly passive ETF) portfolio. I started purchasing NVDA puts simply to protect gains on SPY holdings where I simply want to reduce my Nividia exposure.

raytopia 2y ago

Seeing that AI is being subsidized by investors I wonder how long it'll take for said investors to want a return on their investments and if AI is too expensive for anyone to actually want to use it. Because I don't think a lot of people would pay the actual cost it takes to run AI.

d_silin 2y ago

AI winter is coming! Fascinating how the bust part "boom-bust" cycle invariably comes for every promising tech. On a good side, we have finally have the first generation of AGI. Give it another ten years of improvements before we reach the next AI boom.

chaos_emergent 2y ago

I’m confused by the evidence they use: > to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do Planning and reasoning are the two greatest areas of research in AI right now, with an OOM more researchers devoted to it than there were to the first generation of generative AI architectures > In our experience, even basic summarization tasks often yield illegible and nonsensical results Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs. > even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable We’ve seen a literal 10x decrease in cost from gpt-4-32k to gpt-4o in a single year of AI development (3:1 cost blend). And that ignores that sonnet-3.5 is 50x cheaper than gpt-4-32k while getting better scores on pretty much all benchmarks? > the human brain is 10,000x more effective per unit of power in performing cognitive tasks vs. generative AI Patently false, we’re not untethered brains floating around and require shelter, food, and a ton of other energy intensive requirements to live, and an AI system can perform a task that it is designed to do easily 10-20x faster than a human could. If anything this makes me more bullish about AI systems having a positive ROI; the criticisms they have are based on extraordinarily (if not nefariously) dumb assumptions.

dist-epoch 2y ago
> the human brain is 10,000x more effective per unit of power Even if true, is completely irrelevant. What matters is effectiveness per dolar, not per watt. Does Goldman Sachs have an energy budget per employee, do they hire based on energy consumption? Do they prefer hiring someone living in a small house versus someone living in a mansion with a gigantic energy bill when both request the same salary?
- chaos_emergent 2y ago
  Yeah, an incredibly dumb analysis. I did a back of the hand estimate based off of the following assumptions: gpt-4o: 10 H100s = ~7000W power requirement, 30 seconds to generate a summary for an hour-long call human: ~10.7 kWh/year energy requirement (avg per American), 20 minutes to generate the same summary The numbers come out to 60 Wh for 4o and 400 Wh for a human

sircastor 2y ago

I know that we "want" investment in AI, but I'd be perfectly happy if Wallstreet stays the heck out of this whole situation. Every time investment money gets into a topic - not company, not an industry, a topic - it becomes an absolute mess. Investment feels like a micromanager that won't let you do your work.

newsclues 2y ago

If we replace bankers with AI the returns, the returns may be better.

bparsons 2y ago

Here I am still waiting for all those amazing innovations that the blockchain will bring us.

ben_jones 2y ago

In real time we’ve seen “LLMs are the future” becomethe AI research into AGI is the future”. What’s the next movement of the goal post?

rongenre 2y ago

I'm sort of dreading the next year or so at big corps, in which engineers will hear from their management: "We've made a big investment in AI, and need you to make this work".

sneed_chucker 2y ago

Integration of LLMs with existing compute and data systems will have an excellent return on investment if done correctly. The current brain dead spitball method of shoehorning a chatbot interface on top of every single existing GUI application is not that.

deleted 2y ago

[deleted]

rsynnott 2y ago

Heresy! Kind of a little surprised that they’re coming right out and saying it at this point; I didn’t think we were at that point in the hype cycle just yet.

disqard 2y ago
It might be a bit of a "Prisoner's Dilemma" situation, and by being the first (among hype hawkers) to point out the Emperor's lack of clothing, they're hoping to come out ahead by throwing their competitors under the bus first. If this catches on, it'll mimic the lets-do-layoffs meme spreading virally among tech corps -- IIRC it was Elon's "look I can layoff most of the company and it'll still keep going" which started that trend. Edit: Let's not forget that Goldman Sach will definitely be betting on these outcomes (puts, etc.), so by pushing this narrative, they're definitely going to benefit.