3 ms·
Thank you. There are basically two ways: a) You let the user check the output. If the user says, yay, this meets my needs, great. b) There might be an objectiv
by auggierose 3y ago
Thank you.
There are basically two ways: a) You let the user check the output. If the user says, yay, this meets my needs, great. b) There might be an objective way to check if the output meets a certain objective. For example, you can let the LLM generate the output together with a proof that the output is correct. You can then check the proof separately.
And of course, you can mix and nest a) and b).
- intended 3y agoI meant on the lines of the software stack you were using. Langsmith and Promptflow are what I was considering, however there is precious little discussion yet. There are some great videos for prompt flow though, which is at least one saving grace. Sadly the Human Review option is simply never going to scale - Human review becomes the bottleneck. LLM output is non-deterministic at even temp 0, so scalable solutions are always critical.
- auggierose 3y agoI am just using the OpenAI API directly, and build my own libraries on top of that. Human review can work great depending on how you embed it into your workflow, and how qualified the user is to make a judgement. But yes, the goal is to reduce human review to the specification level.