4 ms·

I've noticed several companies replacing deterministic systems in their support flows with a LLM version that is slower and worse. Many interfaces simply aren't

I've noticed several companies replacing deterministic systems in their support flows with a LLM version that is slower and worse. Many interfaces simply aren't better with AI added

romanovcode 3mo ago
As a contractor who built a lot of predictive systems and workflows in last three years I can tell you that quite often there is a specific request to put AI into it even when it is not needed and would objectively make the system worse, slower and more expensive. The AI psychosis is a real thing.
- QuantumNomad_ 3mo ago
  So then, do you put AI into it anyway because they asked for it, or do you tell them that you won’t do that?
  - romanovcode 3mo ago
    > you tell them that you won’t do that? Of course I will do that, I get paid for doing that. Most of the times I can convince that AI is not necessary by showing small PoC flow with AWS diagrams of data flows. This works well especially if the ask comes from technical people. Other times the C-level interjects (CEO, CFO, sometimes even CTO) and demands that AI should be there. I literally had CEOs send me instagram reels of some AI shovel-sellers to demonstrate that I am wrong and AI is the way to go. No point arguing after that because I have no problem implementing whatever AI they want rather than losing a paying project.
    - nik282000 3mo ago
      Use 'AI' to create anti-AI reels showing how much they suck at all tasks. Spam CEO's underlings.
  - pwndByDeath 3mo ago
    I'm sorry, David, I'm afraid I can't do that.
- cwnyth 3mo ago
  Where I work, management hasn't considered integrating AI at all, yet some clients are very vocal about it being the future and worry we are going to be left behind. Most people just don't care, and I worry the squeaky wheel will eventually get the grease.
  - nutjob2 3mo ago
    > worry we are going to be left behind. I bet lemmings are grateful they were left behind. It beggars belief that people think that they should rush in some uncertain direction, like some drawbridge is going to be lifted the moment people work out what the right direction is. It's utter stupidity.
    - inigyou 3mo ago
      Every single person who bootstrapped becoming powerful did it by rushing into things, but it's a high variance strategy because you could also end up destitute
- KellyCriterion 3mo ago
  Haha, i have a colleague, he is the "AI-is-for-everything-let-me-check-Claude-first": Regardless which task is handed to him, he "discusses" it first with Claude and very often comes back with like "The AI said... X"
  - anal_reactor 3mo ago
    I talk to Claude because I'm very talkative but I have nobody to talk to.
  - Philip-J-Fry 3mo ago
    I have one too. He'll say "Claude says this:" and pastes a screenshot of some Claude Code output. Most of the time it's wrong, or makes assumptions that won't hold true. Or it comes up with some overcomplicated solution and I'm like "This is like a 10 line change, right here". These people just destroy their ability to read and understand the systems they're working with. I actually see it as them making themselves redundant. Because if you can't understand anything without Claude, and Claude doesn't even give the right answers, then what are you worth?
    - KellyCriterion 3mo ago
      ...and now think about Claude being shutdown by the gov... :-D
- pjmlp 3mo ago
  I keep seeing requests to replace what would be a perfect UNIX shell script with agents, like what is the benefit other than being able to say we're doing AI?
- ezst 3mo ago
  Maybe it should have clicked earlier in life and I'm perhaps that much dumb dumb, but it only recently occurred to me (from experiencing it at two very different companies and discussing with peers having reached a certain seniority level more or less at the same time) how dysfunctional many companies are, and how often they produce incentives that are misaligned with the overall company goals and sustainability principles. I blame in large part a layer of middle management that selfishly puts itself above all else, misguides, misrepresents, because it essentially pays larger dividends (literally and not) to "play the networking game than to be an efficient and effective productive structure". Maybe that's to be expected in a services-driven economy where the value of the work is immaterial and subjective (and the whole phenomenon of bullshit jobs).
camdenreslink 3mo ago
The real best case scenario is using LLMs to help build deterministic systems. Instead of asking an LLM to do some task that you know will be repeated, instead ask the LLM to build a program (Python script or whatever) to do the task.
- JCTheDenthog 3mo ago
  Or just write it yourself?
  - dukeyukey 3mo ago
    If you already know what the inputs/outputs are, why should you spend days or weeks of your life typing it out rather than giving it in a well-specified and tested form to an LLM to get it done a hundred times faster?
    - chasd00 3mo ago
      This is a truth that many are having a hard time accepting. Getting shoved into the light so fast is blinding.
      - anon7725 3mo ago
        We understand what you’re claiming, we just think that you are wrong.
    - dosisking 3mo ago
      Because the LLM version will have countless number of bugs and security holes, which means you will spend weeks or months of your life fixing them.
    - skydhash 3mo ago
      Because it’s rarely so black and white. Knowing the inputs and outputs is merely the first steps, you need to think about the transitions too as they have their own costs. Those costs don’t disappear and it’s truly naive to think they don’t matter. Take security issues, they may arise because what you thinks was the input is merely a subset of the true input range. And the extra possibilities lead to unforeseen behavior. A lot of programming is about ensuring that the input and the output are the sets defined in the specs. And the rest is that the transition/relation is the right tradeoffs of performance, correctness, and costs.
al_borland 3mo ago
My management is pushing for us to come up with ideas on where we can use LLMs in our product. The whole team has been very resistant for this exact reason. Anything we can think of will only make things worse, and we’ve already been told anything above a 1-2% failure rate is unacceptable. If anything we need more structure and standards to hit that, not less.
- iwontberude 3mo ago
  Luckily for programmatic or logic following, smaller models tend to do better, it can be surprising at first to see the more expensive models do worse at a task but it’s true.
- rueh 3mo ago
  I believe that llm’s can be used to re-imagine experiences but it’s definitely not the way people think. The constraint is imagination and thinking about complex trade offs more than anything else. Which is the essence of innovation. The agent paradigm will eventually give way to experiences that are a hybrid of deterministic and non deterministic and you won’t even know the llm was involved or visible.
- pjmlp 3mo ago
  We just got dropped into hackatons for having ideas a few weeks ago, AI at all costs, similar feeling.
filup 3mo ago
That's the completely opposite of what people should do. The laborious task of programing logical work flows is the only reason AI is useful for me.
- reg_dunlop 3mo ago
  When I hear about engineers who are bored with coding, I have to imagine it's because the task of "programming logical work flows" has become rote to them. Instead of refining their approach, or challenging their current knowledge base for discovery of inefficiencies or baseless assumptions, they'd rather hit an "easy" button. I understand the desire to NOT do work. I understand the desire to spend quality time and free time with family. And I understand the idea that familiarity breeds contempt. What I don't understand is the willingness to replace a deterministic language/framework/approach with a probabilistic slop machine.
gnuvince 3mo ago
Basically, folks nowadays think that this article[1] was aspirational rather than a cautionary tale. [1] https://thedailywtf.com/articles/Classic-WTF-No-Quack https://thedailywtf.com/articles/Classic-WTF-No-Quack
gedy 3mo ago
With inexperienced or non-technical people, talking to them about AI can be very confusing, as a LOT of their "AI" usecases are basically they didn't realize or know how to write a program for this straightforward logic.
mikert89 3mo ago
models will get smarter, this wont be an issue
- sfn42 3mo ago
  Come talk to me when it isn't an issue.
- throwatdem12311 3mo ago
  They say this every time. Just wait for the NEXT model bro THEN everything will be be fixed. Ok wait maybe not the next one but surely the one after! Hasn’t happened yet and there is no evidence it will.
  - laichzeit0 3mo ago
    This all reminds me of in the 90s when the Borland C++ compilers and Turbo Pascal shit came out and everyone was still hand rolling assembler because the optimising compilers were so bad. I thought Opus 4.6 was pretty good, basically a step change. The stuff I got out of Fable before they blocked it was nearly alien. If things keep improving, I don’t see humans writing code in 2 to 3 years except maybe super niche areas. This will all go the way that optimising compilers did. No amount of resistance, anger or denialism will change that. I’d actually love it if LLMs could skip the slow high level lanaguages entirely and just churned out some weird LLM bytecode that was closer to the metal. I don’t want to read it or understand it at all. Here’s my spec, build it and notify when done. I want to ship stuff not build or dick around with code. Basically like when I go to a shop because I want a table, I don’t care if some carpenter “crafted” it or a machine mass produced and spat it out. It’s cute, but most people just want stuff and don’t care how it’s built.
    - throwatdem12311 3mo ago
      If my experience having to manually unf*ck “production” slop written by sweatshop tier offshore told to go wild with a Claude subscription is any indication then we are a LONG way off from any of that BS. My job is safe because I’m the only person at the company that actually understands what the actual code is doing and I’m the one that gets the calls at 2am and weekends. “Weird LLM bytecode” Why not just generate object code for the target mschine directly?
throwatdem12311 3mo ago
Yeah but did number go up? Can CEO check a box to show investors? Now that’s real value.
Philip-J-Fry 3mo ago
I am seeing similar things in just regular tooling and development. Things that can be solved deterministically or what would have been a simple CLI 5 years ago are now an LLM integration. Instead of using the LLM to create deterministic tools, we are using LLMs to replace them. It's completely backwards and I don't know why people (especially high ranking people in my company at least) seem to think that this is the way forward. No, I don't want a whole CI pipeline that is just LLM prompts. Yes it's very easy, but it's expensive, slow and prone to failure in ways you can't even predict. Same things like using LLMs for the code review process. What would have been a simple linting rule is now a pass with an LLM rather than using the LLM to create the linting rule, which it is absolutely excellent at creating.
- IAmGraydon 3mo ago
  >I am seeing similar things in just regular tooling and development. Yes, and we're also seeing lots of companies claiming they're using "AI" and it's just deterministic under the hood.
sdesol 3mo ago
> replacing deterministic systems in their support flows The issue is, they don't want to provide "better" support but "cheaper" support. Imagine a trained agent that understands the big picture. Now imagine a company investing in humans to use AI to retrieve knowledge that the human can easily identify as being relevant or not, and using that knowledge to better aid the customer. Right now AI is being sold as a "we don't need support personells" instead of "how can we provide better service." For a lot of products, better service will probably not matter as "cheaper" products will win most of the time. Most people don't want to pay for better. They want to pay the same for something better, which is what companies are not investing their time in figuring out how to use AI properly for I think.
- Hendrikto 3mo ago
  > Most people don't want to pay for better. A lot of people want to pay for better, but that is hard. Better is more expensive, most of the time, but being more expensive is no guarantee for being better. It feels like the correlation is very weak. Most expensive products are just expensive, not good. If there was a reliable way to identify the "better" thing, I and a lot of other people would go for that every time we can.
thatjoeoverthr 3mo ago
Unwise design. “It talks, CSRs talk, it’s the same thing”. The fact CSRs talk is incidental. Nobody contacts support to talk. Customer service is a kind of “exception handler” for that which you failed to automate. If your system exists, works and is legible, conversation is avoided.
- ytoawwhra92 3mo ago
  This is not true IME. People constantly call support for problems they can resolve themselves, where the product is actively trying to help them resolve the problem, and where copious documentation exists to communicate exactly how to resolve the problem. For many organisations these calls are >80% of support workload, which is why so many products make you complete a quiz before giving you a support form, or why you have to navigate a phone tree to get a person on the other end. Most recent support call I assisted with was someone who was trying to sign in on a new device, sign in screen told them to check their email for a one-time code and enter it into the form on the screen. They called support because the email took longer than 10 seconds to arrive in their inbox, so they believed the code had already expired by the time it arrived. There was nothing in the form or the email communicating a 10 second code expiry (actual expiry is 60 minutes, not communicated in the form or email). They'd never actually tried to enter the code into the form and hit submit. Never clicked the help link. Just straight to the phone call.

I've noticed several companies replacing deterministic systems in their support flows with a LLM version that is slower and worse. Many interfaces simply aren't better with AI added

romanovcode 3mo ago

As a contractor who built a lot of predictive systems and workflows in last three years I can tell you that quite often there is a specific request to put AI into it even when it is not needed and would objectively make the system worse, slower and more expensive. The AI psychosis is a real thing.

QuantumNomad_ 3mo ago
So then, do you put AI into it anyway because they asked for it, or do you tell them that you won’t do that?
- romanovcode 3mo ago
  > you tell them that you won’t do that? Of course I will do that, I get paid for doing that. Most of the times I can convince that AI is not necessary by showing small PoC flow with AWS diagrams of data flows. This works well especially if the ask comes from technical people. Other times the C-level interjects (CEO, CFO, sometimes even CTO) and demands that AI should be there. I literally had CEOs send me instagram reels of some AI shovel-sellers to demonstrate that I am wrong and AI is the way to go. No point arguing after that because I have no problem implementing whatever AI they want rather than losing a paying project.
  - nik282000 3mo ago
    Use 'AI' to create anti-AI reels showing how much they suck at all tasks. Spam CEO's underlings.
- pwndByDeath 3mo ago
  I'm sorry, David, I'm afraid I can't do that.
cwnyth 3mo ago
Where I work, management hasn't considered integrating AI at all, yet some clients are very vocal about it being the future and worry we are going to be left behind. Most people just don't care, and I worry the squeaky wheel will eventually get the grease.
- nutjob2 3mo ago
  > worry we are going to be left behind. I bet lemmings are grateful they were left behind. It beggars belief that people think that they should rush in some uncertain direction, like some drawbridge is going to be lifted the moment people work out what the right direction is. It's utter stupidity.
  - inigyou 3mo ago
    Every single person who bootstrapped becoming powerful did it by rushing into things, but it's a high variance strategy because you could also end up destitute
KellyCriterion 3mo ago
Haha, i have a colleague, he is the "AI-is-for-everything-let-me-check-Claude-first": Regardless which task is handed to him, he "discusses" it first with Claude and very often comes back with like "The AI said... X"
- anal_reactor 3mo ago
  I talk to Claude because I'm very talkative but I have nobody to talk to.
- Philip-J-Fry 3mo ago
  I have one too. He'll say "Claude says this:" and pastes a screenshot of some Claude Code output. Most of the time it's wrong, or makes assumptions that won't hold true. Or it comes up with some overcomplicated solution and I'm like "This is like a 10 line change, right here". These people just destroy their ability to read and understand the systems they're working with. I actually see it as them making themselves redundant. Because if you can't understand anything without Claude, and Claude doesn't even give the right answers, then what are you worth?
  - KellyCriterion 3mo ago
    ...and now think about Claude being shutdown by the gov... :-D
pjmlp 3mo ago
I keep seeing requests to replace what would be a perfect UNIX shell script with agents, like what is the benefit other than being able to say we're doing AI?
ezst 3mo ago
Maybe it should have clicked earlier in life and I'm perhaps that much dumb dumb, but it only recently occurred to me (from experiencing it at two very different companies and discussing with peers having reached a certain seniority level more or less at the same time) how dysfunctional many companies are, and how often they produce incentives that are misaligned with the overall company goals and sustainability principles. I blame in large part a layer of middle management that selfishly puts itself above all else, misguides, misrepresents, because it essentially pays larger dividends (literally and not) to "play the networking game than to be an efficient and effective productive structure". Maybe that's to be expected in a services-driven economy where the value of the work is immaterial and subjective (and the whole phenomenon of bullshit jobs).

camdenreslink 3mo ago

The real best case scenario is using LLMs to help build deterministic systems. Instead of asking an LLM to do some task that you know will be repeated, instead ask the LLM to build a program (Python script or whatever) to do the task.

JCTheDenthog 3mo ago
Or just write it yourself?
- dukeyukey 3mo ago
  If you already know what the inputs/outputs are, why should you spend days or weeks of your life typing it out rather than giving it in a well-specified and tested form to an LLM to get it done a hundred times faster?
  - chasd00 3mo ago
    This is a truth that many are having a hard time accepting. Getting shoved into the light so fast is blinding.
    - anon7725 3mo ago
      We understand what you’re claiming, we just think that you are wrong.
  - dosisking 3mo ago
    Because the LLM version will have countless number of bugs and security holes, which means you will spend weeks or months of your life fixing them.
  - skydhash 3mo ago
    Because it’s rarely so black and white. Knowing the inputs and outputs is merely the first steps, you need to think about the transitions too as they have their own costs. Those costs don’t disappear and it’s truly naive to think they don’t matter. Take security issues, they may arise because what you thinks was the input is merely a subset of the true input range. And the extra possibilities lead to unforeseen behavior. A lot of programming is about ensuring that the input and the output are the sets defined in the specs. And the rest is that the transition/relation is the right tradeoffs of performance, correctness, and costs.

al_borland 3mo ago

My management is pushing for us to come up with ideas on where we can use LLMs in our product. The whole team has been very resistant for this exact reason. Anything we can think of will only make things worse, and we’ve already been told anything above a 1-2% failure rate is unacceptable. If anything we need more structure and standards to hit that, not less.

iwontberude 3mo ago
Luckily for programmatic or logic following, smaller models tend to do better, it can be surprising at first to see the more expensive models do worse at a task but it’s true.
rueh 3mo ago
I believe that llm’s can be used to re-imagine experiences but it’s definitely not the way people think. The constraint is imagination and thinking about complex trade offs more than anything else. Which is the essence of innovation. The agent paradigm will eventually give way to experiences that are a hybrid of deterministic and non deterministic and you won’t even know the llm was involved or visible.
pjmlp 3mo ago
We just got dropped into hackatons for having ideas a few weeks ago, AI at all costs, similar feeling.

filup 3mo ago

That's the completely opposite of what people should do. The laborious task of programing logical work flows is the only reason AI is useful for me.

reg_dunlop 3mo ago
When I hear about engineers who are bored with coding, I have to imagine it's because the task of "programming logical work flows" has become rote to them. Instead of refining their approach, or challenging their current knowledge base for discovery of inefficiencies or baseless assumptions, they'd rather hit an "easy" button. I understand the desire to NOT do work. I understand the desire to spend quality time and free time with family. And I understand the idea that familiarity breeds contempt. What I don't understand is the willingness to replace a deterministic language/framework/approach with a probabilistic slop machine.

gnuvince 3mo ago

Basically, folks nowadays think that this article[1] was aspirational rather than a cautionary tale. [1] https://thedailywtf.com/articles/Classic-WTF-No-Quack https://thedailywtf.com/articles/Classic-WTF-No-Quack

gedy 3mo ago

With inexperienced or non-technical people, talking to them about AI can be very confusing, as a LOT of their "AI" usecases are basically they didn't realize or know how to write a program for this straightforward logic.

mikert89 3mo ago

models will get smarter, this wont be an issue

sfn42 3mo ago
Come talk to me when it isn't an issue.
throwatdem12311 3mo ago
They say this every time. Just wait for the NEXT model bro THEN everything will be be fixed. Ok wait maybe not the next one but surely the one after! Hasn’t happened yet and there is no evidence it will.
- laichzeit0 3mo ago
  This all reminds me of in the 90s when the Borland C++ compilers and Turbo Pascal shit came out and everyone was still hand rolling assembler because the optimising compilers were so bad. I thought Opus 4.6 was pretty good, basically a step change. The stuff I got out of Fable before they blocked it was nearly alien. If things keep improving, I don’t see humans writing code in 2 to 3 years except maybe super niche areas. This will all go the way that optimising compilers did. No amount of resistance, anger or denialism will change that. I’d actually love it if LLMs could skip the slow high level lanaguages entirely and just churned out some weird LLM bytecode that was closer to the metal. I don’t want to read it or understand it at all. Here’s my spec, build it and notify when done. I want to ship stuff not build or dick around with code. Basically like when I go to a shop because I want a table, I don’t care if some carpenter “crafted” it or a machine mass produced and spat it out. It’s cute, but most people just want stuff and don’t care how it’s built.
  - throwatdem12311 3mo ago
    If my experience having to manually unf*ck “production” slop written by sweatshop tier offshore told to go wild with a Claude subscription is any indication then we are a LONG way off from any of that BS. My job is safe because I’m the only person at the company that actually understands what the actual code is doing and I’m the one that gets the calls at 2am and weekends. “Weird LLM bytecode” Why not just generate object code for the target mschine directly?

throwatdem12311 3mo ago

Yeah but did number go up? Can CEO check a box to show investors? Now that’s real value.

Philip-J-Fry 3mo ago

I am seeing similar things in just regular tooling and development. Things that can be solved deterministically or what would have been a simple CLI 5 years ago are now an LLM integration. Instead of using the LLM to create deterministic tools, we are using LLMs to replace them. It's completely backwards and I don't know why people (especially high ranking people in my company at least) seem to think that this is the way forward. No, I don't want a whole CI pipeline that is just LLM prompts. Yes it's very easy, but it's expensive, slow and prone to failure in ways you can't even predict. Same things like using LLMs for the code review process. What would have been a simple linting rule is now a pass with an LLM rather than using the LLM to create the linting rule, which it is absolutely excellent at creating.

IAmGraydon 3mo ago
>I am seeing similar things in just regular tooling and development. Yes, and we're also seeing lots of companies claiming they're using "AI" and it's just deterministic under the hood.

sdesol 3mo ago

> replacing deterministic systems in their support flows The issue is, they don't want to provide "better" support but "cheaper" support. Imagine a trained agent that understands the big picture. Now imagine a company investing in humans to use AI to retrieve knowledge that the human can easily identify as being relevant or not, and using that knowledge to better aid the customer. Right now AI is being sold as a "we don't need support personells" instead of "how can we provide better service." For a lot of products, better service will probably not matter as "cheaper" products will win most of the time. Most people don't want to pay for better. They want to pay the same for something better, which is what companies are not investing their time in figuring out how to use AI properly for I think.

Hendrikto 3mo ago
> Most people don't want to pay for better. A lot of people want to pay for better, but that is hard. Better is more expensive, most of the time, but being more expensive is no guarantee for being better. It feels like the correlation is very weak. Most expensive products are just expensive, not good. If there was a reliable way to identify the "better" thing, I and a lot of other people would go for that every time we can.

thatjoeoverthr 3mo ago

Unwise design. “It talks, CSRs talk, it’s the same thing”. The fact CSRs talk is incidental. Nobody contacts support to talk. Customer service is a kind of “exception handler” for that which you failed to automate. If your system exists, works and is legible, conversation is avoided.

ytoawwhra92 3mo ago
This is not true IME. People constantly call support for problems they can resolve themselves, where the product is actively trying to help them resolve the problem, and where copious documentation exists to communicate exactly how to resolve the problem. For many organisations these calls are >80% of support workload, which is why so many products make you complete a quiz before giving you a support form, or why you have to navigate a phone tree to get a person on the other end. Most recent support call I assisted with was someone who was trying to sign in on a new device, sign in screen told them to check their email for a one-time code and enter it into the form on the screen. They called support because the email took longer than 10 seconds to arrive in their inbox, so they believed the code had already expired by the time it arrived. There was nothing in the form or the email communicating a 10 second code expiry (actual expiry is 60 minutes, not communicated in the form or email). They'd never actually tried to enter the code into the form and hit submit. Never clicked the help link. Just straight to the phone call.