4 ms·

For customer support I don't think models have gotten better since gpt-4.1. The class of small models, with limited to no reasoning, that need to handle a compl

For customer support I don't think models have gotten better since gpt-4.1. The class of small models, with limited to no reasoning, that need to handle a complex issue with a touch of empathy, has not improved much. I think most are actually worth, as agentic harnesses seem to optimize for solving poorly described problems rather than following complex procedures as written. In other words, instruction following maximizing models seem to make worse free-form agents, but they're really all that some domains need.

jstummbillig 2mo ago
I understand the point (I don't agree with it; tool calling has gotten much better/reliable and that is very important for customer support) but consider: If you can get same for a lot less, that's an improvement. If we found a way to supply fresh water and electricity for -90% cost after 2 years, that would be fantastic. You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.
heaney-555 2mo ago
>The class of small models, with limited to no reasoning What? GPT-4.1 was not a small model! And why wouldn't you use reasoning? You're of course going to see poor results when you restrict yourself to small non-reasoning models, but why would you?
- nostrebored 2mo ago
  "Small" was a poor choice of words here, "low compute budget" is more what I'm getting at. In voice interactions, ttfat is actually relatively important. If you look at models with a <1s ttfat you eliminate almost every reasoning model, less some of the diffusion models and more obscure ddtree/dflash like speculative decoding implementations.
  - heaney-555 2mo ago
    https://openai.com/index/introducing-gpt-live/ https://openai.com/index/introducing-gpt-live/ GPT-Live, which is coming to the API soon, responds instantly while reasoning in the background. So it can say "Hold on, I'll look that up for you" and continue to respond to the user conversationally while running an asynchronous reasoning task in the background. It's not in the API yet, but it should be in the coming weeks. You'll see an enormous improvement compared to GPT-4.1.

jstummbillig 2mo ago

I understand the point (I don't agree with it; tool calling has gotten much better/reliable and that is very important for customer support) but consider: If you can get same for a lot less, that's an improvement. If we found a way to supply fresh water and electricity for -90% cost after 2 years, that would be fantastic. You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.

heaney-555 2mo ago

>The class of small models, with limited to no reasoning What? GPT-4.1 was not a small model! And why wouldn't you use reasoning? You're of course going to see poor results when you restrict yourself to small non-reasoning models, but why would you?

nostrebored 2mo ago
"Small" was a poor choice of words here, "low compute budget" is more what I'm getting at. In voice interactions, ttfat is actually relatively important. If you look at models with a <1s ttfat you eliminate almost every reasoning model, less some of the diffusion models and more obscure ddtree/dflash like speculative decoding implementations.
- heaney-555 2mo ago
  https://openai.com/index/introducing-gpt-live/ https://openai.com/index/introducing-gpt-live/ GPT-Live, which is coming to the API soon, responds instantly while reasoning in the background. So it can say "Hold on, I'll look that up for you" and continue to respond to the user conversationally while running an asynchronous reasoning task in the background. It's not in the API yet, but it should be in the coming weeks. You'll see an enormous improvement compared to GPT-4.1.