3 ms·

Seems like the trick is to use the LLM to produce a query - then the data you get back is simply the result of the query, and you can examine the query to check

Seems like the trick is to use the LLM to produce a query - then the data you get back is simply the result of the query, and you can examine the query to check that it's appropriate.

simonw 4y ago
In my own experiments I've caught ChatGPT running the correct query but then hallucinating the results, because the response was too long for the token context window and got truncated! I have an open issue about that here: https://github.com/simonw/datasette-chatgpt-plugin/issues/2 https://github.com/simonw/datasette-chatgpt-plugin/issues/2 More about my explorations: https://simonwillison.net/2023/Mar/24/datasette-chatgpt-plugin/ https://simonwillison.net/2023/Mar/24/datasette-chatgpt-plug...
netman21 4y ago
This is essentially how we are using ChatGPT to access our database of 3,335 cybersecurity vendors and provide responses.