5 ms·
When the AI gladly accepts orders that cost hundreds of thousands of dollars or 18,000 cups of water, it’s probably not production ready https://youtube.com/sh
by jameslk 1y ago
When the AI gladly accepts orders that cost hundreds of thousands of dollars or 18,000 cups of water, it’s probably not production ready
https://youtube.com/shorts/FDZj6DCWlfc https://youtube.com/shorts/FDZj6DCWlfc
https://www.tiktok.com/@90daygrinder/video/7535508437447298334 https://www.tiktok.com/@90daygrinder/video/75355084374472983... (another example from a different chain)
- sneak 1y agoThis is solved easily by one additional sanity check API call to a different AI. I’m not sure why people think these bugs are like, complete showstopper insurmountable things. It’s a quick fix.
- Zagreus2142 1y agoTaco Bell knows and controls it's own menu and the valid options are already directly encoded in their POS system, including purchase limits. Why would you call out to a different non-deterministic model instead of validating against the complete and deterministic data you have? Taco Bell can afford 1-2 engineers to manage that
- jameslk 1y agoIt seems so, and yet here we are There’s other videos out there (not just of Taco Bell’s implementation per se) of these systems bugging out
- ggghgcdd 1y ago[flagged]
- shakna 1y agoIf Anthropic couldn't achieve that with Project Vend [0], why do you seem to think that everyone else could? > Claudius, believing itself to be a human, told customers it would start delivering products in person, wearing a blue blazer and a red tie. The employees told the AI it couldn’t do that, as it was an LLM with no body. > Alarmed at this information, Claudius contacted the company’s actual physical security — many times — telling the poor guards that they would find him wearing a blue blazer and a red tie standing by the vending machine. [0] https://techcrunch.com/2025/06/28/anthropics-claude-ai-became-a-terrible-business-owner-in-experiment-that-got-weird/ https://techcrunch.com/2025/06/28/anthropics-claude-ai-becam...
- saagarjha 1y agoAIs are not resilient against deliberate attacks, even if you use multiple different models.
- diamond559 1y agoMaybe they shouldn't work w/ customers then, retail workers have to deal w/ hostile customers all the time.
- zamadatix 1y agoThe net result would surely be retail workers only get to deal with hostile (or just difficult, even) customers while the LLM deals with the easy ones. That's what has happened with every other technology introduced to retail - less "business as usual" and "overhead" work and more "oddball" handling. E.g. the electronic PoS and intercom system already have the same kind of effect.
- zamadatix 1y agoThis would be better off if the LLM was used for the human interface but traditional logic was used for the ordering API and its sanity checks. I.e. let it be fine the LLM can bug out on occasion, but keep rigorous boundaries around the amount of risk that's associated with.
- anywhichway 1y agoI feel like we watched different videos.. Seemed like the AI (or other monitoring system) recognized a problem with the 18000 cups of water order and quickly transitioned to a real human. That instance looked pretty production ready to me.
- jameslk 1y agoI interpreted it as the AI system added something strange to the order, and when someone saw it, that’s when the system was cut off. Otherwise the next word sounded like a confirmation That said, this is not the only video floating out there of these type of systems not handling edge cases elegantly
- Dunedan 1y agoI suspect the human worker still had a headset to listen in to the orders at the drive-through and just intervened when she heard that order.
- JKCalhoun 1y agoRegardless, looks like you can't replace everyone with A.I. just yet.
- ggghgcdd 1y ago[flagged]