3 ms·

It's a new tool. Give devs time. Bear in mind that accurate tool use is still a very new capability, one that many devs don't realize exists yet, and one that c

It's a new tool. Give devs time. Bear in mind that accurate tool use is still a very new capability, one that many devs don't realize exists yet, and one that cheap models like GPT-3 have historically struggled with. With time best practices and common patterns will get established.

TeMPOraL 2y ago
I suppose maybe the businesses have different priorities than the users? Wouldn't be the first time. As a user, most of my interactions with pre-ChatGPT breed of chatbots (including "voice bots" on the phone) involved the robot having the capacity to help me at least partially, but failing to understand my request. Fixing that doesn't even require accurate tool use - it requires using LLM on input to parse it in place of whatever they're currently using, instead of directly as a chatbot; something even pre-GPT-3.5 completion models were somewhat good at. I'm all for giving people time with new technologies, but here, 1) somehow they managed to take what should be an out-of-the-box improvement and use it to make things worse, and 2) myself and people like me are on the receiving end of the problem. At scale, there's lots of extra real frustration being created, and lots of additional people-years wasted, through companies jumping on such tools to potentially save a buck by automating away a few jobs. Hell, it probably shows as healthy profit on the books, as it just further disempowers customers, who learn to take a beating instead of fighting, because what's the point anymore...
- mike_hearn 2y ago
  Yeah, I hear you. I think you see using LLMs for parsing only as intuitive and obvious, but I bet most devs don't think of doing it that way, at least not at first. After all, the thing is right there, willing to talk to you, and the most obvious way to use them is to take the words generated and send them to the customer. Using LLMS to generate tool invocations that then generate templated words is a much more indirect use, and frankly up until recently is not always practical due to the amount of context window that all the tool descriptions and possible actions can take up (which must fit not only the 'training' for the model but also the customer's conversation so far). As tool use becomes more widespread and context window sizes go up, I bet we'll start to see blog posts where devs talk about how they made your approach work well, and then others will see it makes sense and start to copy it.

TeMPOraL 2y ago

I suppose maybe the businesses have different priorities than the users? Wouldn't be the first time. As a user, most of my interactions with pre-ChatGPT breed of chatbots (including "voice bots" on the phone) involved the robot having the capacity to help me at least partially, but failing to understand my request. Fixing that doesn't even require accurate tool use - it requires using LLM on input to parse it in place of whatever they're currently using, instead of directly as a chatbot; something even pre-GPT-3.5 completion models were somewhat good at. I'm all for giving people time with new technologies, but here, 1) somehow they managed to take what should be an out-of-the-box improvement and use it to make things worse, and 2) myself and people like me are on the receiving end of the problem. At scale, there's lots of extra real frustration being created, and lots of additional people-years wasted, through companies jumping on such tools to potentially save a buck by automating away a few jobs. Hell, it probably shows as healthy profit on the books, as it just further disempowers customers, who learn to take a beating instead of fighting, because what's the point anymore...

mike_hearn 2y ago
Yeah, I hear you. I think you see using LLMs for parsing only as intuitive and obvious, but I bet most devs don't think of doing it that way, at least not at first. After all, the thing is right there, willing to talk to you, and the most obvious way to use them is to take the words generated and send them to the customer. Using LLMS to generate tool invocations that then generate templated words is a much more indirect use, and frankly up until recently is not always practical due to the amount of context window that all the tool descriptions and possible actions can take up (which must fit not only the 'training' for the model but also the customer's conversation so far). As tool use becomes more widespread and context window sizes go up, I bet we'll start to see blog posts where devs talk about how they made your approach work well, and then others will see it makes sense and start to copy it.