3 ms·
Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the m
by emilsedgh 3y ago
Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful.
Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema.
One problem that is not resolved yet is that it still hallucinates a lot of attributes. For example we have tool that allows it to create contacts in user's CRM. I ask it to:
"Create contacts for top 3 Barcelona players:.
It creates an structure like this"
1. Lionel Messi
- Email: lionel.messi@barcelona.com
- Phone Number: +1234567890
- Tags: Player, Barcelona
2. Gerard Pique
- Email: gerard.pique@barcelona.com
- Phone Number: +1234567891
- Tags: Player, Barcelona
3. Marc-Andre ter Stegen
- Email: marc-terstegen@barcelona.com
- Phone Number: +1234567892
- Tags: Player, Barcelona
And you can see it hallucinated email addresses and phone numbers.
- 037 3y agoI would never rely on an LLM as a source of such information, just as I wouldn't trust the general knowledge of a human being used as a database. Does your workflow include a step for information search? With the new json features, it should be easy to instruct it to perform a search or directly feed it the right pages to parse.
- pluijzer 3y agoChatGPT can be usefully for many things, but you should really, not use it if you want to retrieve factual data. This might partly be resolved by querying the internet like bing does but purely on the language model side these hallucinations are just an unavoidable part of it.
- Spivak 3y agoYep, it's always always write code / query / function / whatever you need that you would parse and retrieve the data from an external system.