3 ms·

Maybe the generated text could be a slightly different colour until it's verified. But you'd have to make sure there's no easy way of verifying everything mindl

• skissane 2y ago
  Technique I've found helpful personally: get the LLM to generate text in small chunks (e.g. a paragraph at a time). After generating each chunk, it is immediately reviewed by a human, who can edit it manually to correct any mistakes, ask the LLM to try again, or prompt the LLM to make specified changes. When the human is satisfied with that chunk, it is saved, and we move on to the next one. Sometimes, the output the LLM generates is correct and I'm just approving it. Other times, it is mostly right, and I can easily identify and correct its errors. Yet other times, it is totally wrong, but often typing out why it is wrong is a good start to actually generating correct text. Often (but not always), the kinds of false assumptions which LLMs make are similar to those a human reader would make, so stuff like "A and B sound very similar but, in the context of this system, actually have completely different meanings" is a useful addition to documentation anyway. The worst case scenario is the LLM generates something which is subtlety wrong, and the human review fails to pick up on the subtle error. But, that's something which can happen even with no LLMs involved at all. It isn't uncommon for people to make subtle errors in documents they write (often because they are misremembering something) and for those subtle errors not to be picked up during the review process. I'm not convinced the odds of this happening with an LLM-assisted workflow are significantly greater than with a purely human workflow.