3 ms·
This is "AGI's finest". It's what we all wanted. Replacing your human support team to be run exclusively by AI LLM bots whilst they hallucinate to their users.
by rvz 2y ago
This is "AGI's finest".
It's what we all wanted. Replacing your human support team to be run exclusively by AI LLM bots whilst they hallucinate to their users. All unchecked.
Now this bug has now turned into a multi-million dollar mistake and costed Cursor to lose millions of dollars overnight.
What if this was a critical control system in a hospital or energy company and their AI support team (with zero humans) hallucinated a wrong meter reading and overcharged their customers? Or the AI support team hallucinated the wrong medication to a patient?
Is this the AGI future we all want?
- n_ary 2y agoAre we sure that the damage is in “millions”? Could it not be just the same old “vocal and loud minority”, who will wake up tomorrow and re-activate their subscription before the lunch break once Cursor team comes and writes a heartfelt apology post (well generated by AI with a system prompt of “you are a world class PR campaign manager …”)?
- falcor84 2y agoYou're right, but this is more general - it has nothing to do with AGI and everything to do with poor management. It reminds me very much of the Chernobyl disaster, and the myriad examples in Taleb's "The Black Swan".
- layer8 2y agoWhere are you seeing AGI?
- aesirian 2y agoIt's a Reddit post with 65 upvotes. Where are people getting millions of dollars overnight? HN is too dramatic lol
- esafak 2y agoEven if they generously compensate locked out users, they'll probably end up ahead using the AI bot.
- deleted 2y ago[deleted]