3 ms·
I built a ChatGPT plugin to answer questions about data hosted in Datasette
- simonw 4y agoThe bad news: I've already caught it hallucinating details despite having looked up the correct information with a SQL query! Some examples in this issue thread: https://github.com/simonw/datasette-chatgpt-plugin/issues/2 https://github.com/simonw/datasette-chatgpt-plugin/issues/2
- simonw 4y agoIn investigating ways to address this I realized it's possible to see the prompts used by other existing integrations - here's the prompt for Wolfram Alpha for example: https://www.wolframalpha.com/.well-known/ai-plugin.json https://www.wolframalpha.com/.well-known/ai-plugin.json
- simonw 4y agoI think it hallucinates when the response returned from a query is too long - if it exceeds the token limit, ChatGPT appears to invisibly truncate it and then hallucinate to make up for the missing data.