3 ms·
I've had some pretty good luck getting OAI assistant API to behave. I have found you won't succeed with a verbose policy in your prompt up-front. You have to s
by bob1029 3y ago
I've had some pretty good luck getting OAI assistant API to behave.
I have found you won't succeed with a verbose policy in your prompt up-front. You have to spoon feed it relevant policies over time as the conversation progresses.
Determining the specific hierarchy and progression of policy seems to be the million dollar part of the question.
I've thrown away almost 10 iterations by now trying to find a stable pattern. There is definitely something here though. I've already seen some of the light.
- PaulHoule 3y agoThe question is: will your luck hold up for someone else?
- bob1029 3y agoPerhaps luck is the wrong word to use. I think there is now a path that offers a degree of statistical certainty that the right thing will happen. People make errors as well. These systems are trained in terms of people the ways they exchange information. I am not seeking perfect. I am seeking reasonable & practical. A non-zero error rate is acceptable to me and my team. My number is 90% right now. I can tolerate a 10% failure rate on the happy path (I.e. cases we expressly intended for it to support). At this target, certain desired functions are not granted yet. Once we reach 99% on the happy path, we will probably consider granting it autonomy ~equivalent to a human employee. Malicious users are out of scope - we aren't serving public traffic. This is employees only and everything is logged and sent for manager review as appropriate.
- PaulHoule 3y agoHow big is your test set?