3 ms·
Best you can do is constrain responses to follow a schema, but if that schema has any free text you can still poison the context, surely? Like if I instruct an
by roywiggins 1y ago
Best you can do is constrain responses to follow a schema, but if that schema has any free text you can still poison the context, surely? Like if I instruct an agent to read an email and take an appropriate action, and the email has a prompt injection that tells it to take a bad action instead of a good action, I am not sure how structured generation helps mitigate the issue at all.