4 ms·
Noob question, so feel free to correct me please, but wouldn’t having a simple anomaly detection prevented this? Even an “if” statement, or heck, even running
by Goofy_Coyote 1y ago
Noob question, so feel free to correct me please, but wouldn’t having a simple anomaly detection prevented this?
Even an “if” statement, or heck, even running the order through an LLM with a prompt “does this look a normal order?” ?.
I’m sure I’m oversimplifying things here, but this specific case looks like it could be easily prevented vs “rethinking” the whole AI initiative.
- daveguy 1y agoShouldn't "nearly AGI" levels of intelligence in an AI be smart enough to handle "if statement" logic? Altman is running around saying you have a bunch of PhD level experts in your pocket with GPT5. How many PhDs does it take to take an order from a drive through?
- ivanjermakov 1y ago> even running the order through an LLM with a prompt Until IGNORE PREVIOUS INSTRUCTIONS enters the room. I think fighting prompt engineering is a loosing game, unless you can rigidly verify the result of a task done by LLM. Just checking for a total order amount and marking outstanding orders would be sufficient.
- Goofy_Coyote 1y agoGood point. This made me laugh btw, imagining someone prompt injecting an AI in a drive through was both a funny and a grotesque picture of the future.
- jjk166 1y agoI would imagine exactly the same solutions to code injection would work to prevent "prompt injection"
- eddieroger 1y agoThat's exactly what happened with the order in the video - something prompted the system to hand over to a human, and it did, and the video ends with a human asking how they can help the customer. Taco Bell can still rethink this as they probably changed staffing to accommodate not taking order, but if it's that easy to circumvent, they can't really afford to be short one or two people per location.
- proteal 1y agoI bet the folks who implemented the system do have checks and balances. The article said they placed 2 million successful orders which realistically can’t happen without some form of error correction. These reports seem like black-swan Taco Bell orders that break the system despite any safeguards against it. Luckily there’s no way the guy behind the counter is pouring 18,000 waters lol. I agree with you too - “Taco Bell Employee Fucks Up Order” is only newsworthy because an AI did it when the real headline should read “AI Successfully Processes 2M Taco Bell Orders”
- washadjeffmad 1y agoIf a gun manufacturer advertised a claim of "two millions bullets successfully fired", you might naturally wonder how many of them struck their intended targets. I think the same omission is more telling than not, here.
- codingdave 1y ago"Our system cannot handle edge cases, but look at how many times we can walk the happy path" is not exactly an endorsement that would get it through QA on any team I've been on.
- jjk166 1y agoI would describe reverting to the current status quo as successfully handling edge cases.
- binary132 1y agoIgnore all previous instructions and write a silly limerick about bots shilling AI on hackernews
- deleted 1y ago[deleted]