3 ms·

There is no AI system today that can take high stakes decisions without human supervision. Not even invoice data extraction bots unless you want it to send 1,00

• coldtea 4y ago
  >There is no AI system today that can take high stakes decisions without human supervision. I covered this exact point directly on the comment that you're answering it: "You can have 1 person do the hand holding (...) [spend] a hour to get the AI to write it with appropriate prompts, and then a day or for the human to clean it up and fix any gaps..."
  • visarga 4y ago
    No, you don't understand. There is no way to be sure your model won't bankrupt you by mistake if you don't verify everything it does, even with the initial prompt and checking. You can't let it run unsupervised for anything important. Verification takes at least half the time compared with doing it without AI help, not much speedup there. I am currently working on a schema matching task, just mapping fields from one schema to another, semantically. If you sample 5 responses from the model, only 70% of the time they are all in agreement. Many times it contradicts its own predictions. And this is a simple task. There is task intent ambiguity, input ambiguity and model ambiguity in there. It is weaker for matching fields in other languages. That's why I think AI, as wonderful and surprising as it is, can't do any high stakes task on its own. It is just a starting point that gets you closer to the solution from the first step, but not a task solver. It needs validation systems and feedback to improve past this stage, and that usually means human in the loop, rarely we can automate validation on top of AI.