8 ms·

When Will the GenAI Bubble Burst?

79 points by isaacfrond 3y ago

aitchnyu 3y ago
Tangential, is the concept of AI Winter still relevant?
- dukeyukey 3y ago
  Of course. Right now we're in an AI Spring, but winter will come again.
- baq 3y ago
  Yeah, nuclear winter and that grey snow which is falling is burnt dollar bills and startups.
- ZiiS 3y ago
  Whilst we are certainly in a hype bubble (AI's third) and there are some indications it may lead to the third AI winter, the two possible outcomes are this bubble doesn't burst, or a future bubble doesn't burst. The long term trend has always been to stronger AI and higher highs.
- pid-1 3y ago
  AI Winter was related to neural networks existing but computers not being fast enough to handle the amount of data needed to train them + massive datasets themselves not existing. So far, I'd say no? New models are still being developed and getting better in very short time frames vs what folks did historically. The finance bubble thesis is more related to: "are companies going to generate revenues to justify the massive investments being done?"
oldpersonintx 3y ago
[dead]
gruturo 3y ago
What if it isn't a bubble? I mean yeah it's overhyped, it's a bit _too_ easy to find capital as long as you slap the AI label on something, but the damn thing _works_ already, you can easily verify this by using it. Current LLMs may be ultimately a dead end and not the road to GenAI, but they are useful already today as they are, and likely will still improve for a while.
- flohofwoe 3y ago
  Being useful in some areas is different from tech companies trying to cram "AI" into every product, no matter if it makes any lick of sense or not. That irrational behaviour is what I would call the real AI bubble, and I can't wait for it to burst ;)
  - gruturo 3y ago
    Agree on that. The need of an AI rice cooker, an AI printer, an AI Juicero, is definitely questionable. Hopefully, eventually, this stuff will run out of money and we'll be left with what makes sense.
- jeffreygoesto 3y ago
  But where is the real use? Chatbots and stock images seem very niche and hallucinations have not been solved either.
  - deleted 3y ago
    [deleted]
  - gruturo 3y ago
    Hallucinations don't matter if you check the output before using it, at least as long as it still saves you plenty of time. Can it find 500 issues in a codebase, 470 of which are real? Good. Can it summarize a conversation and find who proposed a certain course of action? Can it go through 170 years of legal precedent and quote relevant cases (better than the existing, conventional search engines, which you would then later use to verify the quoted cases)?
jascha_eng 3y ago
Glad to see some scepticism. I agree that the development seems to have slowed for a bit. Claude 3 is a bit better than GPT-4 but if we continue with such tiny improvements every year its not enough to keep the hype going imo. GPT-5 has to really slap.
wouldbecouldbe 3y ago
GPT is great and useful. But it reminds me of young kids who are verbally strong. Some 5-year-olds are amazing talkers, giving you the impression they have a very sophisticated intelligence at first glance. But the more time you spend with them, you realize they don't understand but imitate. Although the kids will grow up and understanding naturally develops, many people seem to assume LLMs will take the same path, which seems less likely. Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that. Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing. GenAI will continue to have its purposes, some applications will become more useful with more specific models and more developed reinforcement models on top of it. So there is definitely a commercial use for those. But it will keep getting stuck on the accuracy part, the same place where AGI development is stuck.
- taylorius 3y ago
  That phenomenon goes a lot broader than young kids. Verbal fluency and rhetoric has almost become a sort of personal marketing technique. Sounds impressive at first, but doesn't necessarily correspond to anything of substance behind it. My tentative hypothesis is that it's a symptom of the "optimization" of educational content on the internet.
  - ben_w 3y ago
    Various times I hear someone described as "an idiot's idea of a smart person". I can vibe with that description of GenAI: it does all the things that, as a kid, I regarded as synonymous with intelligent. And yet, now we have it, it's at the level of an intern… an intern in a huge number of fields all at once, so very general and still useful, but an intern nonetheless.
hamilyon2 3y ago
Plateau of productivity is still out there waiting, it is just not in 1 year, as some investors and valuations assume. In 10 years the impact will be so profound LLMs will be everywhere, in 15 years we will not even recognise industry. Mobile boom will be humbled by LLM boom. Of current giants, maybe Microsoft will come out on top, they have everything they need.
- benterix 3y ago
  > Plateau of productivity is still out there waiting Well, actually for me it seems to be there right now. It's not great, it's mediocre, I understand its limits, but I regularly recourse to ChatGPT for simple tasks. I don't trust it, I need to verify each time, but its all factored in. I got to manage around its limitations somehow and it's basically boring tech for me. I guess it's similar for many others.
- yard2010 3y ago
  This is what is going to happen. Recall that most people didn't believe in the internet revolution nor the print revolution when they just started, it feels the same
reissbaker 3y ago
Gary Marcus predicted that deep learning was "hitting a wall" [1]... right before ChatGPT came out. At this point he's so committed to the bit that he has to keep predicting that this time, really, it'll stall, and hoping eventually a stall comes. 1: https://nautil.us/deep-learning-is-hitting-a-wall-238440/ https://nautil.us/deep-learning-is-hitting-a-wall-238440/ Also note he doubts that scaling will work past GPT-3... Not uh, 20/20 foresight exactly.
- soderfoo 3y ago
  When will the “When will the GenAI bubble burst? "bubble burst? I've seen contrarian think pieces like this pop up on social media so much lately they pollute my feed just as much as the GenAI evangelist articles.
  - Hoasi 3y ago
    Maybe it just indicates a general AI buzzwords fatigue, regardless of actual progress in the field.
  - greg_V 3y ago
    That's a signal that the tide IS turning. I follow some people who went all-in on AI (consultants mostly) and their tone has changed quite a bit in the last few months. Cool new tech that was oversold in terms of what it can do for businesses. A correction is in order.
    - AnimalMuppet 3y ago
      A correction is in order in the short term. We don't yet know what ChatGPT can do in the long term. In the short term, though, it can't do as much as it has been sold as being able to do.
- ben_w 3y ago
  Do you have statistics on his error rate, or just examples? I'm of the school of thought one should aim to be about 70-80% right: score above that, and the problems you're tackling aren't hard enough; score below that, and you're not good enough to help with the problems.
jiggawatts 3y ago
This reminds me of the exciting time when I first got to make my own ray-traced 3D graphics... in 1991. I had to sit there and watch as the free POV-Ray tool generated the image pixel by agonising pixel, left-to-right, top-to-bottom. Toy Story was four years later, and required 800,000 machine hours on a "render farm" to produce an output resolution of just 1080p. Now? I get to play games on a single PC at 60 fps with ray tracing at 4K resolution! Conceptually, the glacial pixel-by-pixel rendering in 1991 and the RTX games today are the same thing, but qualitatively there's no comparison. We've been shown that machines can be prompted in English, think about what they've been asked, and produce useful output... at the same kind of rate I was getting from POV-ray in the 90s. Now picture the same thing five years later (equivalent of Toy Story), ten years later (Grand Theft Auto III), twenty years later (Battlefield 3, Kung Fu Panda 2), or thirty years later (Cyberpunk 2077). Some of us can extrapolate from the present and foresee that the pixels crawling one at a time across the screen is not how it's always going to be. That one day the same technology will make our dreams come alive. Others look at nascent technologies like generative AI and think "Pfft... too slow." and write Substack posts.
- rvnx 3y ago
  You mentioned GTA III, now GTA VI that has LLM support... This is going to be the best game of the decade ("full life simulation")
bobm_kite9 3y ago
My pet theory is that OpenAI releases are on hiatus pending the US elections. It would be a huge disaster to release SORA in the next few months and have it some way sway voting. What do you think?
- ZiiS 3y ago
  Sora is so ridiculously expensive, they doubt it has a market.
  - tiborsaas 3y ago
    Not that I doubt it, but curious what's the source for costs?
    - ZiiS 3y ago
      I don't know of any official source. But reading between the lines: Murati also told the Journal that Sora is “much more expensive”. And just how few sample videos they have output. The point of AI generated content is you _never_ need to reuse it, you can just generate more.
- Madmallard 3y ago
  You can’t halt this stuff. There isn’t one player and there is considerable market incentive.
- wilg 3y ago
  Not a chance. Aren't they gearing up for GPT-5 in the next month or two?
taylorius 3y ago
We need that laughing Leonardo Di Caprio meme with Jensen Huang's head pasted on.
colesantiago 3y ago
Just like crypto, AI has recently seen a seismic grift shift in startups and courses all around. Even the VCs know this as the AI industry spent 17x more on nvidia chips than it brought in in revenue. [0] So we will likely see a massive fallout when most of these AI startups are quickly commoditised either by OpenAI, Anthropic etc (better & cheaper AI models), Apple (local AI models) and open source (replicating closed source models that is 10X better) It is an industry that is the race to the cheapest and ultimately the bottom. 0 https://www.wsj.com/tech/ai/a-peter-thiel-backed-ai-startup-cognition-labs-seeks-2-billion-valuation-998fa39d https://www.wsj.com/tech/ai/a-peter-thiel-backed-ai-startup-...
- LordKeren 3y ago
  Unlike crypto, AI already has very strong real world use cases and has been wholly embraced by the public. It’s an unfortunate reality that one of crypto’s long term side effects is instilling a strong tech-cynicism in a large amount of people. This is a consequence of the deluge of bad actors that capitalized on crypto’s inherent ability to enable fraud and speculative “investing” at unprecedented scales.
  - aurareturn 3y ago
    Well said. It's a lazy comparison between crypto and AI. Crypto is a bunch of scammers selling securities in an unregulated market. It exists simply because politicians move too slow. GenAI is more like the invention of electricity or internet. People will figure out ways to build on top of the invention. I'm sure you can go back to the 70s/80s and find that many people have said the bubble would burst or that the technology would never be useful.
    - colesantiago 3y ago
      They are both the same. Both have breathless hype. Both are extremely overvalued and are a solution looking for a problem. Both aren't really useful unless you're (trying to count generating cat memes and broken hallucinating code a use case, it isn't). And last and the most important of all: Both of these waste a ton of energy either by training or running the software and are unlikely to make startups money due to it's rapid commoditization. https://arxiv.org/pdf/2311.16863.pdf https://arxiv.org/pdf/2311.16863.pdf https://www.theverge.com/24066646/ai-electricity-energy-watts-generative-consumption https://www.theverge.com/24066646/ai-electricity-energy-watt...
laborcontract 3y ago
Gary Marcus is not a person writing serious things. We barely have any tooling for this stuff. Most people don’t know still how best to use these things in production workflows. Investors will continue to throw their piles of dry powder at this until a large large company comes out and says “we’ve failed to figure out how to use genai to improve things”. And no company ceo is going to come out soon and say something that bold. Nobody wants to look like a moron. So the answer is, there is no bubble. This is an exploratory phase, and there is a lot more money left to chase opportunities.
hubraumhugo 3y ago
There is one big difference compared to previous hype cycles: While crypto or VR tech still hasn't arrived in our daily lives, most of my friends are already using tools like ChatGPT on a regular basis. Generating images and videos is entertaining, but I believe that the true value of AI lies in automating tedious, repetitive work, both in the digital (e.g. data entry) and physical space (e.g. assembly or logistics work). It's these "boring" use cases that will bring big efficiency boosts to our economy.
- ZiiS 3y ago
  The bubble bust on the Internet too; the outcomes can be wildly different.
  - rvnx 3y ago
    You sure that the bubble bursted ? Tech companies are among the wildest valuations for all-time, including post-burst. If you were QQQ investor in 2000 you are now extensively rich.
    - ZiiS 3y ago
      Most investors are fairly sure the dot-com bubble burst; but it recovered well, and obviously there were still winners. The last two times the AI bubble burst also did not stop us getting here.
- ReleaseCandidat 3y ago
  > There is one big difference compared to previous hype cycles Not if you compare for example to the dot-com bubble.
- rvnx 3y ago
  And it's actually useful. There is a big question to ask yourself when you see a new evolution: would I get sad if this disappears or be missing it ? crypto ? wouldn't care VR ? wouldn't care google glass ? wouldn't care facebook ? wouldn't care internet ? would be a serious step back LLMs ? would be a serious step back etc
em500 3y ago
Probably not until speculators find a new bubble to up. My hunch: GenAI will be very useful embedded in many applications, but mostly in the background (like relational databases and unlike "blockchain technology"). It won't become ubiquitous and ever present, like the internet or mobile phones.
- platelminto 3y ago
  Within a year of launch, nearly all my friends use ChatGPT on a weekly basis. Granted, I'm a university student, so I'm sure I'm in quite a specific bubble, but still - it already feels quite ubiquitous. 100M weekly users too. I don't see how it won't become ever present.
- rsynnott 3y ago
  > Probably not until speculators find a new bubble to up. I mean in that case it's probably ~dead, or at least on life support, within a year or so; speculators are quite good at finding new bubbles.
soderfoo 3y ago
> 50B in, $3B out. That’s obviously not sustainable. Ouch. GenAI seems to promise no shortage of solutions but offers no reliable revenue stream.
dontupvoteme 3y ago
IMO LLMs are basically the best thing since wikipedia for learning/autodidacts. They're also pretty fantastic for discoverability if you do a bit of filtering to remove hallucinations. Only downside is you end up finding that project is abandoned for years or so.
aurareturn 3y ago
Here's one of many example use cases we found for GPT4 API: Our sales people request invoices from a potential customer. On those invoices are our competitor's services and price. Invoices can come in PDF, png, jpeg, excel, csv, email formats. Content formatting can come in random forms. Pricing breakdowns are also non-standard across invoices. We have matching services and our own prices. The goal is to find similar services where we charge less. In the past, our sales people would spend hours combing through those invoices. We wrote a prompt for GPT4, fed in our services and prices, and asked it to find services we could potentially replace as well as our profit margin. It took us a day to write this prompt. The results were outstanding and GPT4 gave accurate results. We even asked it to package it up in a PDF for us to send to the potential customers. On a Monday morning, we started on the prompt. By Tuesday morning, we got it working well enough that we were confident ship it to a few of our sales people to test. This will save our company hundreds of thousands each year and we can get back to the potential customer much faster than before - increasing the likelihood of a sale. If we had to program this like normal software, it'd probably take months to get it right with dedicated engineering resources to account for new invoice edge cases. Chances are, engineering would never even prioritize this feature for our sales people because there is simply no economical way to account for so many different invoice edge cases. I believe we're just getting started. If we get GPT6 in two years and massive improvement in inference cost and context size, it's going to change everything we do. Heck, even GPT4 with 100x context size and 100x lower cost per inference would be transformative. If this is a bubble, I'd like to live in it. I believe that many businesses have found use cases similar to the impact of ours. But they're just not broadcasting them to the internet in order to keep it a business advantage.
- dawidloubser 3y ago
  That's a great use-case! But what about when GPT hallucinates - gets it wrong. What happens when you have sent a (potentially legally-binding) quote to your customers, and you have to back out of it? Humans make mistakes - whether directly, or indirectly through the software that we craft. But when the mistake happens, we can reason about it, understand why it happened, and fix the problem. Wouldn't it be embarrassing to not be in a position to reason about / explain a mistake to your customers caused by this "thing" we use that's been trained on all the random information available on the internet? Would that not reduce the value and standing of your business (which is ultimately about the people and expertise contained within) in the eyes of customers? As an engineer, technologist, and CTO I absolutely love what I've been able to craft using generative AI under direct supervision. But I cannot imagine a world where we give our carefully thought-through, easily changeable, algorithms and processes for a black box that cannot be reasoned about, no matter how good the output _sometimes_ is. I have to think many people feel this way.
olabyne 3y ago
The release of ChatGPT 3, 4 made us astounded because it was doing things a computer never could have done since they exists. That made us believe that they could replace humans in a lot of fields, where, in fact, it only improves the work of an already existing human. The hallucinations, specially, can be very costly. The money saved by doing 95% of a taks with an AI can be swallowed easily by an human having to work to fix the 5%.
- ReptileMan 3y ago
  For developers ChatGPT is like having a team of good juniors on your fingertips. I think it can increase the productivity of a senior engineer probably three fold. But you must do code review and understand the code.
  - mewpmewp2 3y ago
    Yes, I believe it is specifically good when you have a lot of experience so you can immediately tell what is correct, what is wrong. I think this is an excellent time for experienced engs in the market, because GPT might in certain terms hinder low experience engs, but scale exp engs output many times.
iLoveOncall 3y ago
My org at work is organizing a hackathon and the theme is AI-based solutions to help with software development execution. For the past year they've also tried to shoehorn LLMs anywhere they can. Of all the ideas proposed so far, not a single one of them needs LLMs or AI to be done and could be implemented with 30 years old tech. I won't say LLMs aren't more useful than Blockchain, they are, but this is exactly the same situation of all companies trying to find a problem to the "solution" that GenAI is, like they tried to find a problem that Blockchain solves. Except much bigger companies have invested much more money in GenAI. The bubble burst will be spectacular.
scotty79 3y ago
When will railroad bubble burst?
margorczynski 3y ago
I think the main issue is that they look for revenue and income directly generated by AI/ML while this technology (at least currently) is mainly aimed at cost reduction. There might be already tens of billions saved in operating costs for many companies achieved by using "AI" solutions.
physicsguy 3y ago
It won't burst, it'll just eke out over a few years as people realise it's harder to implement than it is to talk about. Some things will be successful, a lot won't. People will move onto the next thing, marketers will market. See: digital twins, cryptocurrencies, semantic web...
jeisc 3y ago
Once AI has been used to launch a successful cyber attack on any vital resource and it gets into the press
redwood 3y ago
All hype cycles involve bubbles and bursts. The point is on a longer time scale the growth happens anyway... look at the .com bubble followed by the internet eating the world. Things just take longer than you expect and initial leaders tend to fall behind. The big question here is where OpenAI and Nvidia will be in 10-15 years... my theory is that chips will be mass produced as we've seen in past waves bringing costs and valuations down for Nvidia... meanwhile OpenAI's death grip from Microsoft will strangle it slowly into an AOL like outcome
benrutter 3y ago
I think GenAI is definitely in a hype phase right now, but even so this take seems out: > $50B in, $3B out. That’s obviously not sustainable There's been a lot of building out of models and tools, presumably the bet is that it'll pay off over the next few years. For what its worth, I'm not convinced it will, but calling it now based on those revenue figures is like saying a ship is a failure after its first voyage because it cost more to build than it made in that single trip.
- aurareturn 3y ago
  Meta spent $100b on VR. $50b is nothing.
vsnf 3y ago
I use GPT-4 to assist in some programming, and it's useful. But I've been struggling to figure out where LLMs make sense as something integrated into a larger product. As an end-user, where I can directly ask them for something specific ("write a function that...") that I can both inspect, double check, and further refine before accepting, they make sense to me. But as an integrated aspect of some system, I haven't been able to come up with any compelling, or even mildly useful usecases. Also, is anyone using the local, non-OpenAI LLMs for anything useful? They've always been slower and measurably worse than OpenAI's GPTs for me.
achow 3y ago
> At least one company must.. > Find some huge problem(s) that they can charge a lot for. > Ship a solution to that problem at a reasonable price. > Whatever they sell must give customers enough confidence to merit adoption. You know which company is this? This is any of the million of companies, who build products based upon customer need and sells with reasonable or great profits - whether it is in space of consumer or enterprise. Pureplay GenAI should only strive to serve these compamnies (via APIs). GenAI companies would have very hard time understanding end users and designing product as per their needs or domain specific workflows. Chat interfaces does not count as product building.
aurareturn 3y ago
One thing we're desperate to do in our team is to somehow feed all of our Slack history into an LLM and ask it questions. There are many decisions, reasons for those decisions, and business logic lost in Slack channels. Slack search isn't nearly good enough to surface them. I can't count how many times I know the answer is in Slack somewhere 5 months ago but I can't find those messages anymore. I see this as an inference problem that is solvable by more efficient LLM software and better/more AI hardware.
- mewpmewp2 3y ago
  I have also thought of doing that, and that can be solved using RAG. Agree that Slack search is terrible. Even just having vector embeddings for search would already be huge.
  - aurareturn 3y ago
    Are there any companies doing this for Slack now? Or are we all just waiting for Slack to add this and charge an extra $20/month per user?
- jascha_eng 3y ago
  You can kinda do this with Notion AI. It's not quite the same as slack ofcourse because you need to manually build up the knowledge base first but if it is well maintained you can use it to answer questions easily. Probably there is some tools out there to create notion pages from slack history? Would be a useful tool at least.
tim333 3y ago
>$50B in, $3B out. That’s obviously not sustainable. Though there will probably be massive value, greater than $50bn in AI in the future. I'm not sure if this plays out like the dot com crash where there was obvious long term value but capital got misallocated to a bunch of pets.coms (https://en.wikipedia.org/wiki/Pets.com https://en.wikipedia.org/wiki/Pets.com) which got wiped out in the crash after 1999 or not. This time most of the $50bn seems to be going to large companies that will probably stick around.
rsynnott 3y ago
> Thus far, the closest thing to a killer app is probably coding assistace Oof, if that is actually the case, then it's in serious trouble, because it isn't actually very good at that.

79 points by isaacfrond 3y ago

aitchnyu 3y ago

Tangential, is the concept of AI Winter still relevant?

dukeyukey 3y ago
Of course. Right now we're in an AI Spring, but winter will come again.
baq 3y ago
Yeah, nuclear winter and that grey snow which is falling is burnt dollar bills and startups.
ZiiS 3y ago
Whilst we are certainly in a hype bubble (AI's third) and there are some indications it may lead to the third AI winter, the two possible outcomes are this bubble doesn't burst, or a future bubble doesn't burst. The long term trend has always been to stronger AI and higher highs.
pid-1 3y ago
AI Winter was related to neural networks existing but computers not being fast enough to handle the amount of data needed to train them + massive datasets themselves not existing. So far, I'd say no? New models are still being developed and getting better in very short time frames vs what folks did historically. The finance bubble thesis is more related to: "are companies going to generate revenues to justify the massive investments being done?"

oldpersonintx 3y ago

[dead]

gruturo 3y ago

What if it isn't a bubble? I mean yeah it's overhyped, it's a bit _too_ easy to find capital as long as you slap the AI label on something, but the damn thing _works_ already, you can easily verify this by using it. Current LLMs may be ultimately a dead end and not the road to GenAI, but they are useful already today as they are, and likely will still improve for a while.

flohofwoe 3y ago
Being useful in some areas is different from tech companies trying to cram "AI" into every product, no matter if it makes any lick of sense or not. That irrational behaviour is what I would call the real AI bubble, and I can't wait for it to burst ;)
- gruturo 3y ago
  Agree on that. The need of an AI rice cooker, an AI printer, an AI Juicero, is definitely questionable. Hopefully, eventually, this stuff will run out of money and we'll be left with what makes sense.
jeffreygoesto 3y ago
But where is the real use? Chatbots and stock images seem very niche and hallucinations have not been solved either.
- deleted 3y ago
  [deleted]
- gruturo 3y ago
  Hallucinations don't matter if you check the output before using it, at least as long as it still saves you plenty of time. Can it find 500 issues in a codebase, 470 of which are real? Good. Can it summarize a conversation and find who proposed a certain course of action? Can it go through 170 years of legal precedent and quote relevant cases (better than the existing, conventional search engines, which you would then later use to verify the quoted cases)?

jascha_eng 3y ago

Glad to see some scepticism. I agree that the development seems to have slowed for a bit. Claude 3 is a bit better than GPT-4 but if we continue with such tiny improvements every year its not enough to keep the hype going imo. GPT-5 has to really slap.

wouldbecouldbe 3y ago

GPT is great and useful. But it reminds me of young kids who are verbally strong. Some 5-year-olds are amazing talkers, giving you the impression they have a very sophisticated intelligence at first glance. But the more time you spend with them, you realize they don't understand but imitate. Although the kids will grow up and understanding naturally develops, many people seem to assume LLMs will take the same path, which seems less likely. Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that. Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing. GenAI will continue to have its purposes, some applications will become more useful with more specific models and more developed reinforcement models on top of it. So there is definitely a commercial use for those. But it will keep getting stuck on the accuracy part, the same place where AGI development is stuck.

taylorius 3y ago
That phenomenon goes a lot broader than young kids. Verbal fluency and rhetoric has almost become a sort of personal marketing technique. Sounds impressive at first, but doesn't necessarily correspond to anything of substance behind it. My tentative hypothesis is that it's a symptom of the "optimization" of educational content on the internet.
- ben_w 3y ago
  Various times I hear someone described as "an idiot's idea of a smart person". I can vibe with that description of GenAI: it does all the things that, as a kid, I regarded as synonymous with intelligent. And yet, now we have it, it's at the level of an intern… an intern in a huge number of fields all at once, so very general and still useful, but an intern nonetheless.

hamilyon2 3y ago

Plateau of productivity is still out there waiting, it is just not in 1 year, as some investors and valuations assume. In 10 years the impact will be so profound LLMs will be everywhere, in 15 years we will not even recognise industry. Mobile boom will be humbled by LLM boom. Of current giants, maybe Microsoft will come out on top, they have everything they need.

benterix 3y ago
> Plateau of productivity is still out there waiting Well, actually for me it seems to be there right now. It's not great, it's mediocre, I understand its limits, but I regularly recourse to ChatGPT for simple tasks. I don't trust it, I need to verify each time, but its all factored in. I got to manage around its limitations somehow and it's basically boring tech for me. I guess it's similar for many others.
yard2010 3y ago
This is what is going to happen. Recall that most people didn't believe in the internet revolution nor the print revolution when they just started, it feels the same

reissbaker 3y ago

Gary Marcus predicted that deep learning was "hitting a wall" [1]... right before ChatGPT came out. At this point he's so committed to the bit that he has to keep predicting that this time, really, it'll stall, and hoping eventually a stall comes. 1: https://nautil.us/deep-learning-is-hitting-a-wall-238440/ https://nautil.us/deep-learning-is-hitting-a-wall-238440/ Also note he doubts that scaling will work past GPT-3... Not uh, 20/20 foresight exactly.

soderfoo 3y ago
When will the “When will the GenAI bubble burst? "bubble burst? I've seen contrarian think pieces like this pop up on social media so much lately they pollute my feed just as much as the GenAI evangelist articles.
- Hoasi 3y ago
  Maybe it just indicates a general AI buzzwords fatigue, regardless of actual progress in the field.
- greg_V 3y ago
  That's a signal that the tide IS turning. I follow some people who went all-in on AI (consultants mostly) and their tone has changed quite a bit in the last few months. Cool new tech that was oversold in terms of what it can do for businesses. A correction is in order.
  - AnimalMuppet 3y ago
    A correction is in order in the short term. We don't yet know what ChatGPT can do in the long term. In the short term, though, it can't do as much as it has been sold as being able to do.
ben_w 3y ago
Do you have statistics on his error rate, or just examples? I'm of the school of thought one should aim to be about 70-80% right: score above that, and the problems you're tackling aren't hard enough; score below that, and you're not good enough to help with the problems.

jiggawatts 3y ago

This reminds me of the exciting time when I first got to make my own ray-traced 3D graphics... in 1991. I had to sit there and watch as the free POV-Ray tool generated the image pixel by agonising pixel, left-to-right, top-to-bottom. Toy Story was four years later, and required 800,000 machine hours on a "render farm" to produce an output resolution of just 1080p. Now? I get to play games on a single PC at 60 fps with ray tracing at 4K resolution! Conceptually, the glacial pixel-by-pixel rendering in 1991 and the RTX games today are the same thing, but qualitatively there's no comparison. We've been shown that machines can be prompted in English, think about what they've been asked, and produce useful output... at the same kind of rate I was getting from POV-ray in the 90s. Now picture the same thing five years later (equivalent of Toy Story), ten years later (Grand Theft Auto III), twenty years later (Battlefield 3, Kung Fu Panda 2), or thirty years later (Cyberpunk 2077). Some of us can extrapolate from the present and foresee that the pixels crawling one at a time across the screen is not how it's always going to be. That one day the same technology will make our dreams come alive. Others look at nascent technologies like generative AI and think "Pfft... too slow." and write Substack posts.

rvnx 3y ago
You mentioned GTA III, now GTA VI that has LLM support... This is going to be the best game of the decade ("full life simulation")

bobm_kite9 3y ago

My pet theory is that OpenAI releases are on hiatus pending the US elections. It would be a huge disaster to release SORA in the next few months and have it some way sway voting. What do you think?

ZiiS 3y ago
Sora is so ridiculously expensive, they doubt it has a market.
- tiborsaas 3y ago
  Not that I doubt it, but curious what's the source for costs?
  - ZiiS 3y ago
    I don't know of any official source. But reading between the lines: Murati also told the Journal that Sora is “much more expensive”. And just how few sample videos they have output. The point of AI generated content is you _never_ need to reuse it, you can just generate more.
Madmallard 3y ago
You can’t halt this stuff. There isn’t one player and there is considerable market incentive.
wilg 3y ago
Not a chance. Aren't they gearing up for GPT-5 in the next month or two?

taylorius 3y ago

We need that laughing Leonardo Di Caprio meme with Jensen Huang's head pasted on.

colesantiago 3y ago

Just like crypto, AI has recently seen a seismic grift shift in startups and courses all around. Even the VCs know this as the AI industry spent 17x more on nvidia chips than it brought in in revenue. [0] So we will likely see a massive fallout when most of these AI startups are quickly commoditised either by OpenAI, Anthropic etc (better & cheaper AI models), Apple (local AI models) and open source (replicating closed source models that is 10X better) It is an industry that is the race to the cheapest and ultimately the bottom. 0 https://www.wsj.com/tech/ai/a-peter-thiel-backed-ai-startup-cognition-labs-seeks-2-billion-valuation-998fa39d https://www.wsj.com/tech/ai/a-peter-thiel-backed-ai-startup-...

LordKeren 3y ago
Unlike crypto, AI already has very strong real world use cases and has been wholly embraced by the public. It’s an unfortunate reality that one of crypto’s long term side effects is instilling a strong tech-cynicism in a large amount of people. This is a consequence of the deluge of bad actors that capitalized on crypto’s inherent ability to enable fraud and speculative “investing” at unprecedented scales.
- aurareturn 3y ago
  Well said. It's a lazy comparison between crypto and AI. Crypto is a bunch of scammers selling securities in an unregulated market. It exists simply because politicians move too slow. GenAI is more like the invention of electricity or internet. People will figure out ways to build on top of the invention. I'm sure you can go back to the 70s/80s and find that many people have said the bubble would burst or that the technology would never be useful.
  - colesantiago 3y ago
    They are both the same. Both have breathless hype. Both are extremely overvalued and are a solution looking for a problem. Both aren't really useful unless you're (trying to count generating cat memes and broken hallucinating code a use case, it isn't). And last and the most important of all: Both of these waste a ton of energy either by training or running the software and are unlikely to make startups money due to it's rapid commoditization. https://arxiv.org/pdf/2311.16863.pdf https://arxiv.org/pdf/2311.16863.pdf https://www.theverge.com/24066646/ai-electricity-energy-watts-generative-consumption https://www.theverge.com/24066646/ai-electricity-energy-watt...

laborcontract 3y ago

Gary Marcus is not a person writing serious things. We barely have any tooling for this stuff. Most people don’t know still how best to use these things in production workflows. Investors will continue to throw their piles of dry powder at this until a large large company comes out and says “we’ve failed to figure out how to use genai to improve things”. And no company ceo is going to come out soon and say something that bold. Nobody wants to look like a moron. So the answer is, there is no bubble. This is an exploratory phase, and there is a lot more money left to chase opportunities.

hubraumhugo 3y ago

There is one big difference compared to previous hype cycles: While crypto or VR tech still hasn't arrived in our daily lives, most of my friends are already using tools like ChatGPT on a regular basis. Generating images and videos is entertaining, but I believe that the true value of AI lies in automating tedious, repetitive work, both in the digital (e.g. data entry) and physical space (e.g. assembly or logistics work). It's these "boring" use cases that will bring big efficiency boosts to our economy.

ZiiS 3y ago
The bubble bust on the Internet too; the outcomes can be wildly different.
- rvnx 3y ago
  You sure that the bubble bursted ? Tech companies are among the wildest valuations for all-time, including post-burst. If you were QQQ investor in 2000 you are now extensively rich.
  - ZiiS 3y ago
    Most investors are fairly sure the dot-com bubble burst; but it recovered well, and obviously there were still winners. The last two times the AI bubble burst also did not stop us getting here.
ReleaseCandidat 3y ago
> There is one big difference compared to previous hype cycles Not if you compare for example to the dot-com bubble.
rvnx 3y ago
And it's actually useful. There is a big question to ask yourself when you see a new evolution: would I get sad if this disappears or be missing it ? crypto ? wouldn't care VR ? wouldn't care google glass ? wouldn't care facebook ? wouldn't care internet ? would be a serious step back LLMs ? would be a serious step back etc

em500 3y ago

Probably not until speculators find a new bubble to up. My hunch: GenAI will be very useful embedded in many applications, but mostly in the background (like relational databases and unlike "blockchain technology"). It won't become ubiquitous and ever present, like the internet or mobile phones.

platelminto 3y ago
Within a year of launch, nearly all my friends use ChatGPT on a weekly basis. Granted, I'm a university student, so I'm sure I'm in quite a specific bubble, but still - it already feels quite ubiquitous. 100M weekly users too. I don't see how it won't become ever present.
rsynnott 3y ago
> Probably not until speculators find a new bubble to up. I mean in that case it's probably ~dead, or at least on life support, within a year or so; speculators are quite good at finding new bubbles.

soderfoo 3y ago

> 50B in, $3B out. That’s obviously not sustainable. Ouch. GenAI seems to promise no shortage of solutions but offers no reliable revenue stream.

dontupvoteme 3y ago

IMO LLMs are basically the best thing since wikipedia for learning/autodidacts. They're also pretty fantastic for discoverability if you do a bit of filtering to remove hallucinations. Only downside is you end up finding that project is abandoned for years or so.

aurareturn 3y ago

Here's one of many example use cases we found for GPT4 API: Our sales people request invoices from a potential customer. On those invoices are our competitor's services and price. Invoices can come in PDF, png, jpeg, excel, csv, email formats. Content formatting can come in random forms. Pricing breakdowns are also non-standard across invoices. We have matching services and our own prices. The goal is to find similar services where we charge less. In the past, our sales people would spend hours combing through those invoices. We wrote a prompt for GPT4, fed in our services and prices, and asked it to find services we could potentially replace as well as our profit margin. It took us a day to write this prompt. The results were outstanding and GPT4 gave accurate results. We even asked it to package it up in a PDF for us to send to the potential customers. On a Monday morning, we started on the prompt. By Tuesday morning, we got it working well enough that we were confident ship it to a few of our sales people to test. This will save our company hundreds of thousands each year and we can get back to the potential customer much faster than before - increasing the likelihood of a sale. If we had to program this like normal software, it'd probably take months to get it right with dedicated engineering resources to account for new invoice edge cases. Chances are, engineering would never even prioritize this feature for our sales people because there is simply no economical way to account for so many different invoice edge cases. I believe we're just getting started. If we get GPT6 in two years and massive improvement in inference cost and context size, it's going to change everything we do. Heck, even GPT4 with 100x context size and 100x lower cost per inference would be transformative. If this is a bubble, I'd like to live in it. I believe that many businesses have found use cases similar to the impact of ours. But they're just not broadcasting them to the internet in order to keep it a business advantage.

dawidloubser 3y ago
That's a great use-case! But what about when GPT hallucinates - gets it wrong. What happens when you have sent a (potentially legally-binding) quote to your customers, and you have to back out of it? Humans make mistakes - whether directly, or indirectly through the software that we craft. But when the mistake happens, we can reason about it, understand why it happened, and fix the problem. Wouldn't it be embarrassing to not be in a position to reason about / explain a mistake to your customers caused by this "thing" we use that's been trained on all the random information available on the internet? Would that not reduce the value and standing of your business (which is ultimately about the people and expertise contained within) in the eyes of customers? As an engineer, technologist, and CTO I absolutely love what I've been able to craft using generative AI under direct supervision. But I cannot imagine a world where we give our carefully thought-through, easily changeable, algorithms and processes for a black box that cannot be reasoned about, no matter how good the output _sometimes_ is. I have to think many people feel this way.

olabyne 3y ago

The release of ChatGPT 3, 4 made us astounded because it was doing things a computer never could have done since they exists. That made us believe that they could replace humans in a lot of fields, where, in fact, it only improves the work of an already existing human. The hallucinations, specially, can be very costly. The money saved by doing 95% of a taks with an AI can be swallowed easily by an human having to work to fix the 5%.

ReptileMan 3y ago
For developers ChatGPT is like having a team of good juniors on your fingertips. I think it can increase the productivity of a senior engineer probably three fold. But you must do code review and understand the code.
- mewpmewp2 3y ago
  Yes, I believe it is specifically good when you have a lot of experience so you can immediately tell what is correct, what is wrong. I think this is an excellent time for experienced engs in the market, because GPT might in certain terms hinder low experience engs, but scale exp engs output many times.

iLoveOncall 3y ago

My org at work is organizing a hackathon and the theme is AI-based solutions to help with software development execution. For the past year they've also tried to shoehorn LLMs anywhere they can. Of all the ideas proposed so far, not a single one of them needs LLMs or AI to be done and could be implemented with 30 years old tech. I won't say LLMs aren't more useful than Blockchain, they are, but this is exactly the same situation of all companies trying to find a problem to the "solution" that GenAI is, like they tried to find a problem that Blockchain solves. Except much bigger companies have invested much more money in GenAI. The bubble burst will be spectacular.

scotty79 3y ago

When will railroad bubble burst?

margorczynski 3y ago

I think the main issue is that they look for revenue and income directly generated by AI/ML while this technology (at least currently) is mainly aimed at cost reduction. There might be already tens of billions saved in operating costs for many companies achieved by using "AI" solutions.

physicsguy 3y ago

It won't burst, it'll just eke out over a few years as people realise it's harder to implement than it is to talk about. Some things will be successful, a lot won't. People will move onto the next thing, marketers will market. See: digital twins, cryptocurrencies, semantic web...

jeisc 3y ago

Once AI has been used to launch a successful cyber attack on any vital resource and it gets into the press

redwood 3y ago

All hype cycles involve bubbles and bursts. The point is on a longer time scale the growth happens anyway... look at the .com bubble followed by the internet eating the world. Things just take longer than you expect and initial leaders tend to fall behind. The big question here is where OpenAI and Nvidia will be in 10-15 years... my theory is that chips will be mass produced as we've seen in past waves bringing costs and valuations down for Nvidia... meanwhile OpenAI's death grip from Microsoft will strangle it slowly into an AOL like outcome

benrutter 3y ago

I think GenAI is definitely in a hype phase right now, but even so this take seems out: > $50B in, $3B out. That’s obviously not sustainable There's been a lot of building out of models and tools, presumably the bet is that it'll pay off over the next few years. For what its worth, I'm not convinced it will, but calling it now based on those revenue figures is like saying a ship is a failure after its first voyage because it cost more to build than it made in that single trip.

aurareturn 3y ago
Meta spent $100b on VR. $50b is nothing.

vsnf 3y ago

I use GPT-4 to assist in some programming, and it's useful. But I've been struggling to figure out where LLMs make sense as something integrated into a larger product. As an end-user, where I can directly ask them for something specific ("write a function that...") that I can both inspect, double check, and further refine before accepting, they make sense to me. But as an integrated aspect of some system, I haven't been able to come up with any compelling, or even mildly useful usecases. Also, is anyone using the local, non-OpenAI LLMs for anything useful? They've always been slower and measurably worse than OpenAI's GPTs for me.

achow 3y ago

> At least one company must.. > Find some huge problem(s) that they can charge a lot for. > Ship a solution to that problem at a reasonable price. > Whatever they sell must give customers enough confidence to merit adoption. You know which company is this? This is any of the million of companies, who build products based upon customer need and sells with reasonable or great profits - whether it is in space of consumer or enterprise. Pureplay GenAI should only strive to serve these compamnies (via APIs). GenAI companies would have very hard time understanding end users and designing product as per their needs or domain specific workflows. Chat interfaces does not count as product building.

aurareturn 3y ago

One thing we're desperate to do in our team is to somehow feed all of our Slack history into an LLM and ask it questions. There are many decisions, reasons for those decisions, and business logic lost in Slack channels. Slack search isn't nearly good enough to surface them. I can't count how many times I know the answer is in Slack somewhere 5 months ago but I can't find those messages anymore. I see this as an inference problem that is solvable by more efficient LLM software and better/more AI hardware.

mewpmewp2 3y ago
I have also thought of doing that, and that can be solved using RAG. Agree that Slack search is terrible. Even just having vector embeddings for search would already be huge.
- aurareturn 3y ago
  Are there any companies doing this for Slack now? Or are we all just waiting for Slack to add this and charge an extra $20/month per user?
jascha_eng 3y ago
You can kinda do this with Notion AI. It's not quite the same as slack ofcourse because you need to manually build up the knowledge base first but if it is well maintained you can use it to answer questions easily. Probably there is some tools out there to create notion pages from slack history? Would be a useful tool at least.

tim333 3y ago

>$50B in, $3B out. That’s obviously not sustainable. Though there will probably be massive value, greater than $50bn in AI in the future. I'm not sure if this plays out like the dot com crash where there was obvious long term value but capital got misallocated to a bunch of pets.coms (https://en.wikipedia.org/wiki/Pets.com https://en.wikipedia.org/wiki/Pets.com) which got wiped out in the crash after 1999 or not. This time most of the $50bn seems to be going to large companies that will probably stick around.

rsynnott 3y ago

> Thus far, the closest thing to a killer app is probably coding assistace Oof, if that is actually the case, then it's in serious trouble, because it isn't actually very good at that.