3 ms·

This experiment used Opus 4.6. Customer service bots typically are not using frontier models.

• johndhi 4mo ago
  Gemini says: "It would cost approximately $6.25 to $30.00 to have Claude Opus 4.6 respond to 10,000 emails, assuming a typical 200-word input and 50-word output per email."
  • cuchoi 4mo ago
    You need to add Openclaw's system prompt and instructions (and the times I had to re read emails multiple times due to multiple issues that happened during the competition :))
  • antonvs 4mo ago
    Gemini is often terrible with that sort of prediction. I've been optimizing an ML training pipeline using Gemini, and it regularly confidently tells me that some optimization will cut training time down to 3 hours. The reality: nothing has run in less than 11 hours so far, and even that's only at the cost of reduced model accuracy. It's helpful with the actual technical changes needed, it just has no concept of what they translate to in the real world. Btw my company is spending > $100/day in relatively cheap Gemini tokens for this work. It's easy to see why one might want to be cautious about exposing a token-burning service to the internet.