3 ms·
We don’t know “what we are seeing” because we are looking from the outside. That’s my point. We can see a chat bot and we can see bad behavior and there are cle
by dpark 5mo ago
We don’t know “what we are seeing” because we are looking from the outside. That’s my point. We can see a chat bot and we can see bad behavior and there are clearly a lot of assumptions that the problem is that someone gave the bot a set of general tools and a prompt and it went off the rails. And that is a possible scenario. It’s also possible that they stuck a dumb chatbot in front of an existing automated account reclamation flow that worked exactly this way but no one noticed.
Do we actually know that a human was in the loop before and that the human judgement was replaced by an LLM? Or is that pure speculation?
I have certainly seen account reclamation flows that allowed providing a new email address (but usually with better safeguards).
- acdha 5mo agoWe know that Meta made a big deal about how they were moving all support to AI: https://www.meta.com/account-recovery-support/ai-support-assistant/ https://www.meta.com/account-recovery-support/ai-support-ass... Now, it’s possible that they instead moved it to human workers and simultaneously forgot everything they’d learned about security or training, but that seems unlikely.