7 ms·

> We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it mo

by Aeolun 1y ago

> We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.

misnome 1y ago
“Let’s deploy something as or more error prone as Brad at infinite scale across our organisation”
candiddevmike 1y ago
I don't understand why generative AI gets a pass at constantly being wrong, but an average worker would be fired if they performed the same way. If a manager needed to constantly correct you or double check your work, you'd be out. Why are we lowering the bar for generative AI?
- amscanne 1y ago
  It’s much cheaper than Brenda (superficially, at least). I’m not sure a worker that costs a few dollars a day would be fired, especially given the occasional brilliance they exhibit.
- anon721656321 1y ago
  If a worker could be right 50% of the time and get paid 1 cent to write a 5000 word essay on a random topic, and do it in less than 30 seconds. Then I think managers would be fine hiring that worker for that rate as well.
  - cryptonym 1y ago
    5000 half-right words is worthless output. That can even lead to negative productivity.
    - dr-detroit 1y ago
      [dead]
  - hitarpetar 1y ago
    great, now who are you paying to sort the right output from the wrong output?
- Esophagus4 1y ago
  Because it doesn’t have to be as accurate as a human to be a helpful tool. That is precisely why we have humans in the loop for so many AI applications. If [AI + human reviewer to correct it] is some multiple more efficient than [human alone], there is still plenty of value.
  - bigstrat2003 1y ago
    > Because it doesn’t have to be as accurate as a human to be a helpful tool. I disagree. If something can't be as accurate as a (good) human, then it's useless to me. I'll just ask the human instead, because I know that the human is going to be worth listening to.
    - Esophagus4 1y ago
      Autopilot in airplanes is a good example to disprove that. Good in most conditions. Not as good as a human. Which is why we still have skilled pilots flying planes, assisted by autopilot. We don’t say “it’s not as good as a human, so stuff it.” We say, “it’s great in most conditions. And humans are trained how to leverage it effectively and trained to fly when it cannot be used.”
      - sjsdaiuasgdia 1y ago
        The autopilots in aircraft have predictable behaviors based on the data and inputs available to them. This can still be problematic! If sensors are feeding the autopilot bad data, the autopilot may do the wrong thing for a situation. Likewise, if the pilot(s) do not understand the autopilot's behaviors, they may misuse the autopilot, or take actions that interfere with the autopilot's operation. Generative AI has unpredictable results. You cannot make confident statements like "if inputs X, Y, and Z are at these values, the system will always produce this set of outputs". In the very short timeline of reacting to a critical mid-flight situation, confidence in the behavior of the systems is critical. A lot of plane crashes have "the pilot didn't understand what the automation was doing" as a significant contributing factor. We get enough of that from lack of training, differences between aircraft manufacturers, and plain old human fallibility. We don't need to introduce a randomized source of opportunities for the pilots to not understand what the automation is doing.
- martin-t 1y ago
  Because it's much cheaper. So now you don't have to pay people to do their actual work, you assign the work to ML ("AI") and then pay the people to check what it generated. That's a very different task, menial and boring, but if it produces more value for the same amount of input money, then it's economical to do so. And since checking the output is often a lower skilled job, you can even pay the people less, pocketing more as an owner.
- Levitz 1y ago
  There's a variety of reasons. You don't have a human to manage. The relationship is completely one-sided, you can query a generative AI at 3 in the morning on new years eve. This entity has no emotions to manage and no own interests. There's cost. There's an implicit promise of improvement over time. There's an the domain of expertise being inhumanly wide. You can ask about cookies right now, then about XII century France, then about biochemistry. The fact that an average worker would be fired if they perform the same way is what the human actually competes with. They have responsibility, which is not something AI can offer. If it was the case that, say, Anthropic, actually signed contracts stating that they are liable for any mistakes, then humans would be absolutely toast.
- BeFlatXIII 1y ago
  How much compute costs is it for the AI to do Brenda's job? Not total AI spend, but the fraction that replaced Brenda. That's why they'd fire a human but keep using the AI.
  - simonw 1y ago
    Brenda has been kissed on her forehead by the Excel goddess herself. She is irreplaceable. (More seriously, she also has 20+ years of institutional knowledge about how the company works, none of which has ever been captured anywhere else.)
  - mrgoldenbrown 1y ago
    It's not just compute, its also the setup costs - How much did you have to pay someone to feed the AI Brenda's decades of knowledge specific to her company and all the little special cases of how it does business.
  - Covenant0028 1y ago
    Brenda's job involves being accountable for the output. In many types of jobs, posting false numbers would render her liable for a dismissal, lawsuit, or even jail. I'd like to see the cost of a model where the model provider (Anthropic etc) can assume that kind of financial and legal accountability. To the extent that this output is possible only when Anthropic is not held to the same standard as Brenda, we will need to conclude that the cost savings accrue due to the reduced liability standards than on the technical capabilities of the model
- basscomm 1y ago
  My kneejerk reaction is the sunk cost fallacy (AI is expensive), but I'm pretty sure it's actually because businesses have spent the last couple of decades doing absolutely everything they can to automate as many humans out of the workforce as possible.
- ryandrake 1y ago
  I've been trying to open my mind and "give AI a chance" lately. I spent all day yesterday struggling with Claude Code's utter incompetence. It behaves worse than any junior engineer I've ever worked with: - It says it's done when its code does not even work, sometimes when it does not even compile. - When asked to fix a bug, it confidently declares victory without actually having fixed the bug. - It gets into this mode where, when it doesn't know what to do, it just tries random things over and over, each time confidently telling me "Perfect! I found the error!" and then waiting for the inevitable response from me: "No, you didn't. Revert that change". - Only when you give it explicit, detailed commands, "modify fade_output to be -90," will it actually produce decent results, but by the time I get to that level of detail, I might as well be writing the code myself. To top it off, unlike the junior engineer, Claude never learns from its mistakes. It makes the same ones over and over and over, even if you include "don't make XYZ mistake" in the prompt. If I were an eng manager, Claude would be on a PIP.
  - simonw 1y ago
    Learning to use Claude Code (and similar coding agents) effectively takes quite a lot of work. Did you have it creating and running automated tests as it worked?
    - 9rx 1y ago
      > Learning to use Claude Code (and similar coding agents) effectively takes quite a lot of work. I've tried to put in the work. I can even get it working well for a while. But then all of a sudden it is like the model suffers a massive blow to the head and can't produce anything coherent anymore. Then it is back to the drawing board, trying all over again. It is exhausting. The promise of what it could be is really tempting fruit, but I am at the point that I can't find the value. The cost of my time to put in the work is not being multiplied in return. > Did you have it creating and running automated tests as it worked? Yes. I work in a professional capacity. This is a necessity regardless of who (or what) is producing the product.
  - hitarpetar 1y ago
    yOu'Re HoLdInG iT wRoNg
- deleted 1y ago
  [deleted]
- latchup 1y ago
  Multiple reasons: * Gen AI never disagrees with or objects to boss's ideas, even if they are bad or harmful to the company or others. In fact, it always praises them no matter what. Brenda, being a well-intentioned human being, might object to bad or immoral ideas to prevent harm. Since boss's ego is too fragile to accept criticism, he prefers gen AI. * Boss is usually not qualified, willing, or free to do Brenda's job to the same quality standard as Brenda. This compels him to pay Brenda and treat her with basic decency, which is a nuisance. Gen AI does not demand fair or decent treatment and (at least for now) is cheaper than Brenda. It can work at any time and under conditions Brenda refuses to. So boss prefers gen AI. * Brenda takes accountability for and pride in her work, making sure it is of high quality and as free of errors as she can manage. This is wasteful: boss only needs output that is good enough to make it someone else's problem, and as fast as possible. This is exactly what gen AI gives him, so boss prefers gen AI.
  - thewarrior 1y ago
    You’re absolutely right !
  - bigfishrunning 1y ago
    Your third point is especially poignant. It points out that AI doesn't just take a job here, but it makes everything worse. I wish I could upvote this comment twice.
- danans 1y ago
  > Why are we lowering the bar for generative AI? Because it doesn't need to sleep or spend time with its family.
- Covenant0028 1y ago
  Gen AI doesn't just get a pass at being wrong. It gets a pass for everything. Look at Grok. If a human employee went around sexually harassing their CEO in public and giving themselves a Hitler nickname, they'd be fired immediately and have criminal charges. In the case of Grok, the CEO had to quit the company after being sexually harassed. We've not lowered the bar for AI, we've removed it entirely.
conductr 1y ago
It’s not even greater trust. It’s just passive trust. The thing is, Brenda is her own QA department. Every good Brenda is precisely good because she checks her own work before shipping it. AI does not do this. It doesn’t even fully understand the problem/question sometimes yet provides a smart definitive sounding answer. It’s like the doctor on The Simpson’s, if you can’t tell he’s a quack, you probably would follow his medical advice.
- dionian 1y ago
  Brenda + AI > Brenda
  - conductr 1y ago
    That’s definitely the hype. But I don’t know if I agree. I’m essentially a Brenda in my corporate finance job and so far have struggled to find any useful scenarios to use AI for. I thought once this can build me a Gantt chart because that’s an annoying task in excel. I had the data. When I asked it to help me, “I can’t do that but I can summarize your data”. Not helpful. Any type of analysis is exactly what I don’t want to trust it with. But I could use help actually building things, which it wouldn’t do. Also, Brenda’s are usually fast. Having them use a tool like AI that can’t be fully trusted just slows them down. So IMO, we haven’t proven the AI variable in your equation is actually a positive value.
    - wat10000 1y ago
      I can't speak to finance. In programming, it can be useful but it takes some time and effort to find where it works well. I have had no success in using it to create production code. It's just not good enough. It tends to pattern-match the problem in somewhat broad strokes and produce something that looks good but collapses if you dig into it. It might work great for CRUD apps but my work is a lot more fiddly than that. I've had good success in using it to create one-off helper scripts to analyze data or test things. For code that doesn't have to be good and doesn't have to stand the test of time, it can do alright. I've had great success in having it do relatively simple analysis on large amounts of code. I see a bug that involves X, and I know that it's happening in Y. There's no immediately obvious connection between X and Y. I can dig into the codebase and trace the connection. Or I can ask the machine to do it. The latter is a hundred times faster. The key is finding things where it can produce useful results and you can verify them quickly. If it says X and Y are connected by such-and-such path and here's how that triggers the bug, I can go look at the stuff and see if that's actually true. If it is, I've saved a lot of time. If it isn't, no big loss. If I ask it to make some one-off data analysis script, I can evaluate the script and spot-check the results and have some confidence. If I ask it to modify some complicated multithreaded code, it's not likely to get it right, and the effort it takes to evaluate its output is way too much for it to be worthwhile.
mrgoldenbrown 1y ago
They want to trust it, because then they can stop paying Brenda, save a few dollars, and buy a 3rd yacht.
m463 1y ago
> No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda. I would add a little nuance here. I know a lot of people who don't have technical ability either because they advanced out of hands-on or never had it because it wasn't their job/interest. These types of people are usually the folks who set direction or govern the purse strings. here's the thing: They are empowered by AI. they can do things themselves. and every one of them is so happy. They are tickled pink.

by Aeolun 1y ago

> We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.

misnome 1y ago

“Let’s deploy something as or more error prone as Brad at infinite scale across our organisation”

candiddevmike 1y ago

I don't understand why generative AI gets a pass at constantly being wrong, but an average worker would be fired if they performed the same way. If a manager needed to constantly correct you or double check your work, you'd be out. Why are we lowering the bar for generative AI?

amscanne 1y ago
It’s much cheaper than Brenda (superficially, at least). I’m not sure a worker that costs a few dollars a day would be fired, especially given the occasional brilliance they exhibit.
anon721656321 1y ago
If a worker could be right 50% of the time and get paid 1 cent to write a 5000 word essay on a random topic, and do it in less than 30 seconds. Then I think managers would be fine hiring that worker for that rate as well.
- cryptonym 1y ago
  5000 half-right words is worthless output. That can even lead to negative productivity.
  - dr-detroit 1y ago
    [dead]
- hitarpetar 1y ago
  great, now who are you paying to sort the right output from the wrong output?
Esophagus4 1y ago
Because it doesn’t have to be as accurate as a human to be a helpful tool. That is precisely why we have humans in the loop for so many AI applications. If [AI + human reviewer to correct it] is some multiple more efficient than [human alone], there is still plenty of value.
- bigstrat2003 1y ago
  > Because it doesn’t have to be as accurate as a human to be a helpful tool. I disagree. If something can't be as accurate as a (good) human, then it's useless to me. I'll just ask the human instead, because I know that the human is going to be worth listening to.
  - Esophagus4 1y ago
    Autopilot in airplanes is a good example to disprove that. Good in most conditions. Not as good as a human. Which is why we still have skilled pilots flying planes, assisted by autopilot. We don’t say “it’s not as good as a human, so stuff it.” We say, “it’s great in most conditions. And humans are trained how to leverage it effectively and trained to fly when it cannot be used.”
    - sjsdaiuasgdia 1y ago
      The autopilots in aircraft have predictable behaviors based on the data and inputs available to them. This can still be problematic! If sensors are feeding the autopilot bad data, the autopilot may do the wrong thing for a situation. Likewise, if the pilot(s) do not understand the autopilot's behaviors, they may misuse the autopilot, or take actions that interfere with the autopilot's operation. Generative AI has unpredictable results. You cannot make confident statements like "if inputs X, Y, and Z are at these values, the system will always produce this set of outputs". In the very short timeline of reacting to a critical mid-flight situation, confidence in the behavior of the systems is critical. A lot of plane crashes have "the pilot didn't understand what the automation was doing" as a significant contributing factor. We get enough of that from lack of training, differences between aircraft manufacturers, and plain old human fallibility. We don't need to introduce a randomized source of opportunities for the pilots to not understand what the automation is doing.
martin-t 1y ago
Because it's much cheaper. So now you don't have to pay people to do their actual work, you assign the work to ML ("AI") and then pay the people to check what it generated. That's a very different task, menial and boring, but if it produces more value for the same amount of input money, then it's economical to do so. And since checking the output is often a lower skilled job, you can even pay the people less, pocketing more as an owner.
Levitz 1y ago
There's a variety of reasons. You don't have a human to manage. The relationship is completely one-sided, you can query a generative AI at 3 in the morning on new years eve. This entity has no emotions to manage and no own interests. There's cost. There's an implicit promise of improvement over time. There's an the domain of expertise being inhumanly wide. You can ask about cookies right now, then about XII century France, then about biochemistry. The fact that an average worker would be fired if they perform the same way is what the human actually competes with. They have responsibility, which is not something AI can offer. If it was the case that, say, Anthropic, actually signed contracts stating that they are liable for any mistakes, then humans would be absolutely toast.
BeFlatXIII 1y ago
How much compute costs is it for the AI to do Brenda's job? Not total AI spend, but the fraction that replaced Brenda. That's why they'd fire a human but keep using the AI.
- simonw 1y ago
  Brenda has been kissed on her forehead by the Excel goddess herself. She is irreplaceable. (More seriously, she also has 20+ years of institutional knowledge about how the company works, none of which has ever been captured anywhere else.)
- mrgoldenbrown 1y ago
  It's not just compute, its also the setup costs - How much did you have to pay someone to feed the AI Brenda's decades of knowledge specific to her company and all the little special cases of how it does business.
- Covenant0028 1y ago
  Brenda's job involves being accountable for the output. In many types of jobs, posting false numbers would render her liable for a dismissal, lawsuit, or even jail. I'd like to see the cost of a model where the model provider (Anthropic etc) can assume that kind of financial and legal accountability. To the extent that this output is possible only when Anthropic is not held to the same standard as Brenda, we will need to conclude that the cost savings accrue due to the reduced liability standards than on the technical capabilities of the model
basscomm 1y ago
My kneejerk reaction is the sunk cost fallacy (AI is expensive), but I'm pretty sure it's actually because businesses have spent the last couple of decades doing absolutely everything they can to automate as many humans out of the workforce as possible.
ryandrake 1y ago
I've been trying to open my mind and "give AI a chance" lately. I spent all day yesterday struggling with Claude Code's utter incompetence. It behaves worse than any junior engineer I've ever worked with: - It says it's done when its code does not even work, sometimes when it does not even compile. - When asked to fix a bug, it confidently declares victory without actually having fixed the bug. - It gets into this mode where, when it doesn't know what to do, it just tries random things over and over, each time confidently telling me "Perfect! I found the error!" and then waiting for the inevitable response from me: "No, you didn't. Revert that change". - Only when you give it explicit, detailed commands, "modify fade_output to be -90," will it actually produce decent results, but by the time I get to that level of detail, I might as well be writing the code myself. To top it off, unlike the junior engineer, Claude never learns from its mistakes. It makes the same ones over and over and over, even if you include "don't make XYZ mistake" in the prompt. If I were an eng manager, Claude would be on a PIP.
- simonw 1y ago
  Learning to use Claude Code (and similar coding agents) effectively takes quite a lot of work. Did you have it creating and running automated tests as it worked?
  - 9rx 1y ago
    > Learning to use Claude Code (and similar coding agents) effectively takes quite a lot of work. I've tried to put in the work. I can even get it working well for a while. But then all of a sudden it is like the model suffers a massive blow to the head and can't produce anything coherent anymore. Then it is back to the drawing board, trying all over again. It is exhausting. The promise of what it could be is really tempting fruit, but I am at the point that I can't find the value. The cost of my time to put in the work is not being multiplied in return. > Did you have it creating and running automated tests as it worked? Yes. I work in a professional capacity. This is a necessity regardless of who (or what) is producing the product.
- hitarpetar 1y ago
  yOu'Re HoLdInG iT wRoNg
deleted 1y ago
[deleted]
latchup 1y ago
Multiple reasons: * Gen AI never disagrees with or objects to boss's ideas, even if they are bad or harmful to the company or others. In fact, it always praises them no matter what. Brenda, being a well-intentioned human being, might object to bad or immoral ideas to prevent harm. Since boss's ego is too fragile to accept criticism, he prefers gen AI. * Boss is usually not qualified, willing, or free to do Brenda's job to the same quality standard as Brenda. This compels him to pay Brenda and treat her with basic decency, which is a nuisance. Gen AI does not demand fair or decent treatment and (at least for now) is cheaper than Brenda. It can work at any time and under conditions Brenda refuses to. So boss prefers gen AI. * Brenda takes accountability for and pride in her work, making sure it is of high quality and as free of errors as she can manage. This is wasteful: boss only needs output that is good enough to make it someone else's problem, and as fast as possible. This is exactly what gen AI gives him, so boss prefers gen AI.
- thewarrior 1y ago
  You’re absolutely right !
- bigfishrunning 1y ago
  Your third point is especially poignant. It points out that AI doesn't just take a job here, but it makes everything worse. I wish I could upvote this comment twice.
danans 1y ago
> Why are we lowering the bar for generative AI? Because it doesn't need to sleep or spend time with its family.
Covenant0028 1y ago
Gen AI doesn't just get a pass at being wrong. It gets a pass for everything. Look at Grok. If a human employee went around sexually harassing their CEO in public and giving themselves a Hitler nickname, they'd be fired immediately and have criminal charges. In the case of Grok, the CEO had to quit the company after being sexually harassed. We've not lowered the bar for AI, we've removed it entirely.

conductr 1y ago

It’s not even greater trust. It’s just passive trust. The thing is, Brenda is her own QA department. Every good Brenda is precisely good because she checks her own work before shipping it. AI does not do this. It doesn’t even fully understand the problem/question sometimes yet provides a smart definitive sounding answer. It’s like the doctor on The Simpson’s, if you can’t tell he’s a quack, you probably would follow his medical advice.

dionian 1y ago
Brenda + AI > Brenda
- conductr 1y ago
  That’s definitely the hype. But I don’t know if I agree. I’m essentially a Brenda in my corporate finance job and so far have struggled to find any useful scenarios to use AI for. I thought once this can build me a Gantt chart because that’s an annoying task in excel. I had the data. When I asked it to help me, “I can’t do that but I can summarize your data”. Not helpful. Any type of analysis is exactly what I don’t want to trust it with. But I could use help actually building things, which it wouldn’t do. Also, Brenda’s are usually fast. Having them use a tool like AI that can’t be fully trusted just slows them down. So IMO, we haven’t proven the AI variable in your equation is actually a positive value.
  - wat10000 1y ago
    I can't speak to finance. In programming, it can be useful but it takes some time and effort to find where it works well. I have had no success in using it to create production code. It's just not good enough. It tends to pattern-match the problem in somewhat broad strokes and produce something that looks good but collapses if you dig into it. It might work great for CRUD apps but my work is a lot more fiddly than that. I've had good success in using it to create one-off helper scripts to analyze data or test things. For code that doesn't have to be good and doesn't have to stand the test of time, it can do alright. I've had great success in having it do relatively simple analysis on large amounts of code. I see a bug that involves X, and I know that it's happening in Y. There's no immediately obvious connection between X and Y. I can dig into the codebase and trace the connection. Or I can ask the machine to do it. The latter is a hundred times faster. The key is finding things where it can produce useful results and you can verify them quickly. If it says X and Y are connected by such-and-such path and here's how that triggers the bug, I can go look at the stuff and see if that's actually true. If it is, I've saved a lot of time. If it isn't, no big loss. If I ask it to make some one-off data analysis script, I can evaluate the script and spot-check the results and have some confidence. If I ask it to modify some complicated multithreaded code, it's not likely to get it right, and the effort it takes to evaluate its output is way too much for it to be worthwhile.

mrgoldenbrown 1y ago

They want to trust it, because then they can stop paying Brenda, save a few dollars, and buy a 3rd yacht.

m463 1y ago

> No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda. I would add a little nuance here. I know a lot of people who don't have technical ability either because they advanced out of hands-on or never had it because it wasn't their job/interest. These types of people are usually the folks who set direction or govern the purse strings. here's the thing: They are empowered by AI. they can do things themselves. and every one of them is so happy. They are tickled pink.