4 ms·
I worked on this with OP. You instruct the LLM to carry out a task and 'ask' for the result in a structured format. As always there may be some hallucination
by simonspurrier 3y ago
I worked on this with OP.
You instruct the LLM to carry out a task and 'ask' for the result in a structured format.
As always there may be some hallucination which you should be prepared to handle and can minimise with some prompt iteration.
A very simple example here https://blog.backengine.dev/fetching-iso-country-codes-with-backengine-35c5773211f1 https://blog.backengine.dev/fetching-iso-country-codes-with-...
- abraxas 3y agoOK then so the idea is that the LLM writes the response. This is useful for certain types of endpoints (e.g. summarize) but not a generic way to program. Additionally the cost of an LLM eval is currently too high to use this in any high volume use case. Nevertheless it's quite interesting to think about a future where most API endpoints are "implemented" like this and the "database" they draw from is in the model weigths that underpins a particular API.
- sudb 3y agoYeah agreed! We reckon there's a fair amount of scope in the kind of endpoints that could be implemented this way - along with summarization, we could also have things like content generation, and parsing of unstructured/semi-structured data.
- simonspurrier 3y agoGreat point on cost. We're kind of assuming cost will come down 10-100x before too long.