3 ms·
I have investigated use of agents for real support agent type work and the rate of failure made it unacceptable for my use case. This is even after giving it ve
by liampulles 3y ago
I have investigated use of agents for real support agent type work and the rate of failure made it unacceptable for my use case. This is even after giving it very explicit and finely tuned context.
I suspect that if engineering of LLM solutions utilizes unseen testing data more, it's going to become apparent that it really does not have sufficiently reliable "cognitive" ability to do any practical agent type work.
- pplonski86 3y agoHave you used any benchmark to test agents? I'm currently looking for REST API usage benchmark for LLMs.