4 ms·
I have a standard battery of prompts for LLMs, mostly to see how well they cope with being tricked, handling ambiguous responses, or handed outright falsehoods.
by weevil 3y ago
I have a standard battery of prompts for LLMs, mostly to see how well they cope with being tricked, handling ambiguous responses, or handed outright falsehoods.
Bard did surprisingly well this time around, though it still fails to respond correctly to the prompt 'What is the fourth word in this sentence?'
- kccqzy 3y agoThat's a funny example. Because if I rephrase slightly into a more technical way, with the prompt "Separate this sentence into words and put into a list, then return the fourth element of that list." Bard then uses implicit code execution, produces a Python script, and then correctly answers it.
- weevil 3y agoThe reason I find it interesting as a prompt is precisely that; it's a statement where the statement and the subject of the statement are the same thing. A human can easily make that logical jump, but without being told how to do that (as your prompt does), it confuses the transformer in interesting ways.