6 ms·

Pivot to AI: Hallucinations worsen as the money runs out

47 points by awfulsystems 2y ago

zer00eyz 2y ago
FTA "Unfortunately, the venture-capital-funded AI industry runs on the promise of replacing humans with a very large shell script — including in areas where details matter. If the AI’s output is just plausible nonsense, that’s a problem. So the hallucination issue is causing a slight panic among AI company leadership." I mean... If the VC's start funding AI companies with crypto is it incest or cannibalism. Can someone come up with an idea for computing that doesn't involve wasting electricity with GPU's on pictures of cats and dogs?
- smackeyacky 2y ago
  We are about to find out if the “bullshit jobs” theory was correct. I suspect it is, we could easily replace the flibbertigibbet class of agile project managers with 3 lines of bash (including comments) and my own manager is a poorly formed excel macro.
Olesya000 2y ago
[dead]
Havoc 2y ago
> the systems don’t have a path to profitable functionality. I don’t buy this at all. Sure loads of ChatGPT wrappers will die but there is definitely money to be made. Robotics is going to be big. Companies are firing customer service people. Etc Sure that last one is perhaps not desirable but it does prove both practical value and feasibility And as the price per token plummets and context increases more application will open up. Just because VC money is keeping some shops open that should die doesn’t mean that’s true for the entire field
- otabdeveloper4 2y ago
  > Robotics is going to be big. You mean "sexbots"? Too niche and expensive. For industrial automation you don't need "AI". > Companies are firing customer service people. They don't need "AI" to do this.
  - Havoc 2y ago
    Robotics as in figure AI. Not sure how you got to sexbots.. > They don't need "AI" to do this. No but it sure helps: https://www.euronews.com/next/2024/02/28/klarnas-ai-bot-is-doing-the-work-of-700-employees-what-will-happen-to-their-jobs https://www.euronews.com/next/2024/02/28/klarnas-ai-bot-is-d...
    - otabdeveloper4 2y ago
      Chatbots isn't "robotics".
      - Havoc 2y ago
        Agreed. It’s also not even close to what I said but yeah sure
- danudey 2y ago
  > Companies are firing customer service people. And they're starting to reap the rewards: https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/?sh=67722f31696f https://www.forbes.com/sites/marisagarcia/2024/02/19/what-ai... AI customer service chatbots are just as prone to hallucination as any other AI chatbot, and companies are finding out that they're liable when their AIs lie to their customers. As we see more companies replacing humans with AIs, we're going to start to see more companies being found liable for the claims that their AIs spun out of whole cloth. We haven't seen an AI yet which can consistently reproduce the truth, rather than something that sounds a lot like something that could be the truth. When someone makes an AI which is reliably just a knowledge engine, which can take an input, process it, and accurately convey the knowledge available in that input, then that will be a killer app; when an AI can take a company's documentation and summarize and condense it for easier consumption, that AI will be insanely popular in business cases both internal and external. Until then, these customer-service-replacing AI chatbots are going to prove to be more and more of a maintenance and liability headache.
bidder33 2y ago
having exhausted the number of people he can sell his “crypto bad” books to, gerard has now pivoted to “ai bad”. nothing new. https://pessimistsarchive.org/ https://pessimistsarchive.org/
- __loam 2y ago
  You're right, there was absolutely nothing to be pessimistic about vis a vis crypto. Nothing bad ever happened there.
- lxgr 2y ago
  To be fair, it doesn't look like he was far off on crypto. Of course being able to call that doesn't at all automatically translate into also being right about AI. There's lots of people that have successfully predicted one particular bubble, but all of their other predictions have completely failed to materialize (to say nothing of all the wrong ones they made before striking gold). That's just how selection/survivorship bias works. In this particular case, I actually suspect that both things can be true at the same time: We're in the middle of an AI hype bubble and >90% of current startups will not survive the next 3 years, but AI will still profoundly change the way we work. There's even a nice historical analogy: We once had the dot com bubble, in which lots of people lost lots of money, but ultimately, the Internet did turn out to be kind of a big deal. It's just hard to go from that general bet on vibes to one that identifies actual companies that are going to bring about all of that.
  - __loam 2y ago
    I think you're right, but I don't think "AI" as we see it today is really the thing we should be looking at when we come out of this bubble. What's really exciting is that we're building an enormous parallel computing infrastructure that's going to revolutionize scientific computing. We're already seeing some glimmers of this stuff like with what deepmind did with protein folding. Once the charletans die off I hope we see a massive acceleration in research in all kinds of fields.
    - lxgr 2y ago

47 points by awfulsystems 2y ago

FTA "Unfortunately, the venture-capital-funded AI industry runs on the promise of replacing humans with a very large shell script — including in areas where details matter. If the AI’s output is just plausible nonsense, that’s a problem. So the hallucination issue is causing a slight panic among AI company leadership." I mean... If the VC's start funding AI companies with crypto is it incest or cannibalism. Can someone come up with an idea for computing that doesn't involve wasting electricity with GPU's on pictures of cats and dogs?

smackeyacky 2y ago
We are about to find out if the “bullshit jobs” theory was correct. I suspect it is, we could easily replace the flibbertigibbet class of agile project managers with 3 lines of bash (including comments) and my own manager is a poorly formed excel macro.

Olesya000 2y ago

[dead]

Havoc 2y ago

> the systems don’t have a path to profitable functionality. I don’t buy this at all. Sure loads of ChatGPT wrappers will die but there is definitely money to be made. Robotics is going to be big. Companies are firing customer service people. Etc Sure that last one is perhaps not desirable but it does prove both practical value and feasibility And as the price per token plummets and context increases more application will open up. Just because VC money is keeping some shops open that should die doesn’t mean that’s true for the entire field

otabdeveloper4 2y ago
> Robotics is going to be big. You mean "sexbots"? Too niche and expensive. For industrial automation you don't need "AI". > Companies are firing customer service people. They don't need "AI" to do this.
- Havoc 2y ago
  Robotics as in figure AI. Not sure how you got to sexbots.. > They don't need "AI" to do this. No but it sure helps: https://www.euronews.com/next/2024/02/28/klarnas-ai-bot-is-doing-the-work-of-700-employees-what-will-happen-to-their-jobs https://www.euronews.com/next/2024/02/28/klarnas-ai-bot-is-d...
  - otabdeveloper4 2y ago
    Chatbots isn't "robotics".
    - Havoc 2y ago
      Agreed. It’s also not even close to what I said but yeah sure
danudey 2y ago
> Companies are firing customer service people. And they're starting to reap the rewards: https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/?sh=67722f31696f https://www.forbes.com/sites/marisagarcia/2024/02/19/what-ai... AI customer service chatbots are just as prone to hallucination as any other AI chatbot, and companies are finding out that they're liable when their AIs lie to their customers. As we see more companies replacing humans with AIs, we're going to start to see more companies being found liable for the claims that their AIs spun out of whole cloth. We haven't seen an AI yet which can consistently reproduce the truth, rather than something that sounds a lot like something that could be the truth. When someone makes an AI which is reliably just a knowledge engine, which can take an input, process it, and accurately convey the knowledge available in that input, then that will be a killer app; when an AI can take a company's documentation and summarize and condense it for easier consumption, that AI will be insanely popular in business cases both internal and external. Until then, these customer-service-replacing AI chatbots are going to prove to be more and more of a maintenance and liability headache.

bidder33 2y ago

having exhausted the number of people he can sell his “crypto bad” books to, gerard has now pivoted to “ai bad”. nothing new. https://pessimistsarchive.org/ https://pessimistsarchive.org/

__loam 2y ago
You're right, there was absolutely nothing to be pessimistic about vis a vis crypto. Nothing bad ever happened there.
lxgr 2y ago
To be fair, it doesn't look like he was far off on crypto. Of course being able to call that doesn't at all automatically translate into also being right about AI. There's lots of people that have successfully predicted one particular bubble, but all of their other predictions have completely failed to materialize (to say nothing of all the wrong ones they made before striking gold). That's just how selection/survivorship bias works. In this particular case, I actually suspect that both things can be true at the same time: We're in the middle of an AI hype bubble and >90% of current startups will not survive the next 3 years, but AI will still profoundly change the way we work. There's even a nice historical analogy: We once had the dot com bubble, in which lots of people lost lots of money, but ultimately, the Internet did turn out to be kind of a big deal. It's just hard to go from that general bet on vibes to one that identifies actual companies that are going to bring about all of that.
- __loam 2y ago
  I think you're right, but I don't think "AI" as we see it today is really the thing we should be looking at when we come out of this bubble. What's really exciting is that we're building an enormous parallel computing infrastructure that's going to revolutionize scientific computing. We're already seeing some glimmers of this stuff like with what deepmind did with protein folding. Once the charletans die off I hope we see a massive acceleration in research in all kinds of fields.
  - lxgr 2y ago