4 ms·

AI agents fail tasks 70% of the time

• rogerkirkness 1y ago
  Agents went from 10% to 30% reliable this year, which is still a big deal.
  • bogzz 1y ago
    lol
• JTbane 1y ago
  "We test baseline agents powered by both closed API-based and open-weights language models (LMs), and find that the most competitive agent can complete 30% of tasks autonomously."
• gavinray 1y ago
  So you ask it to try every task 3.33 times for guaranteed success?
• thebigspacefuck 1y ago
  This is from a Dec 2024 which feels like a while ago
  • creatonez 1y ago
    Hardly anything changed
• bsallthewaydown 1y ago
  AI is a going to be the next bubble. It can't even figure out who the real author of a sculpture is. It's really all BS made up to play with markets and geopolitics. Enjoy it while it lasts.
• drannex 1y ago
  Yes! but, when they work, they only kinda work, sort of.