I doubt it was by design. It's still an autocomplete machine in the end. Claude tries to do agents better than anyone and part of this was to see where it would

I doubt it was by design. It's still an autocomplete machine in the end. Claude tries to do agents better than anyone and part of this was to see where it would fail. There's plenty of blind spots in a real world situation. It was trained to answer questions and score well in exams, not to negotiate and evaluate the value of a tungsten cube, and so on. There weren't enough scams in the training set. And seeing how rare data is from a CEO's perspective, I expect it would perform much worse. It's not just vending machines, it does poorly in games with full wikis too.