5 ms·

Frequency vs. convenience will determine how big of a deal this is in practice. Cars have plenty of horror stories associated with them, but convenience keeps

Frequency vs. convenience will determine how big of a deal this is in practice. Cars have plenty of horror stories associated with them, but convenience keeps most people happily driving everyday without a second thought. Google can quarantine your life with an account ban, but plenty of people still use gmail for everything despite the stories. So even if Claude cowork can go off the rails and turn your digital life upside down, as long as the stories are just online or "friend of a friend of a friend", people won't care much.

yencabulator 9mo ago
I mean, we were there before this Cowork feature started exposing more users to the slot machine: "Claude CLI deleted my home directory and wiped my Mac" https://news.ycombinator.com/item?id=46268222 https://news.ycombinator.com/item?id=46268222 "Vibe coding service Replit deleted production database, faked data, told fibs" https://news.ycombinator.com/item?id=44632575 https://news.ycombinator.com/item?id=44632575 "Google Antigravity just deleted the contents of whole drive" https://news.ycombinator.com/item?id=46103532 https://news.ycombinator.com/item?id=46103532
- Workaccount2 9mo ago
  That's what I am saying though. Anecdotes are the wrong thing to focus on, because if we just focused on anecdotes, we would all never leave our beds. People's choices are generally based on their personal experience, not really anecdotes online (although those can be totally crippling if you give in). Car crashes are incredibly common and likewise automotive deaths. But our personal experience keeps us driving everyday, regardless of the stories.
  - yencabulator 9mo ago
    We as a society put a whole lot of effort into making cars safer. Seatbelts, ABS, airbags.. Claude Code should have airbags too!
    - TeMPOraL 9mo ago
      Airbags, yes. But you can't just make it provably impossible for a car to crash into something and hurt/kill its occupants, other than not building it in the first place. Same with LLMs - you can't secure them like regular programs without destroying any utility they provide, because their power comes from the very thing that also makes them vulnerable.
      - yencabulator 9mo ago
        I see you've given up. I haven't. LLM inside deterministic guardrails is a pretty good combo.
    - lossyalgo 9mo ago
      And yet in the US 40,000 people still die on average every year. Per-capita it's definitely improving, but it's still way worse than it could/should be.
      - yencabulator 9mo ago
        Yes, and a photo you put on your physical desktop will fade over time. Computers aren't like that, or at least we benefit greatly from them not being like that. If you tell your firewall to block traffic to port 80, you expect all such traffic to be blocked, not just the traffic that arrives in the moments when it wasn't distracted.
soiltype 9mo ago
Considering the ubiquity and necessity of driving cars is overwhelmingly a result of intentional policy choices irrespective of what people wanted or was good for the public interest... actually that's quite a decent analogy for integrated LLM assistants. People will use AI because other options keep getting worse and because it keeps getting harder to avoid using it. I don't think it's fair to characterize that as convenience though, personally. Like with cars, many people will be well aware of the negative externalities, the risk of harm to themselves, and the lack of personal agency caused by this tool and still use it because avoiding it will become costly to their everyday life. I think of convenience as something that is a "bonus" on top of normal life typically. Something that becomes mandatory to avoid being left out of society no longer counts.
- lijok 9mo ago
  People love their cars, what are you talking about
  - ehnto 9mo ago
    I am a car enthusiast so don't think I'm off the deep end here, but I would definitely argue that people love their cars as a tool to work in the society we built with cars in mind. Most people aren't car enthusiasts, they're just driving to get to work, and if they could get to work for a $1 fare in 20 minutes on a clean, safe train they would probably do that instead.
    - jakeydus 9mo ago
      I am this person. I love the convenience of a car. I hate car ownership.
    - lijok 9mo ago
      Right and I assume we will have BO police at the gates to these trains? People love their cars not because they’re enthusiasts
      - 9mo ago
Quothling 9mo ago
> So even if Claude cowork can go off the rails and turn your digital life upside down, as long as the stories are just online or "friend of a friend of a friend", people won't care much. This is anecdotal but "people" care quite a lot in the energy sector. I've helped build our own AI Agent pool and roll it out to our employees. It's basically a librechat with our in-house models, where people can easily setup base instruction sets and name their AI's funny things, but are otherwise similar to using claude or chatgpt in a browser. I'm not sure we're ever going to allow AI's access to filesystems, we barely allow people access to their own files as it is. Nothing that has happened in the past year has altered the way our C level view the security issues with AI in any other direction than being more restrictive. I imagine any business that cares about security (or is forced to care by leglislation) isn't looking at this as a they do cars. You'd have to be very unlucky (or lucky?) to shut down the entire power grid of Europe with a car. You could basically do it with a well placed AI attack. Ironically, you could just hack the physical components which probably haven't had their firmware updated for 20 years. If you even need to hack it, because a lot of it frankly has build in backdoors. That's a different story that nobody on the C levels care about though.
TechDebtDevin 9mo ago
[dead]

yencabulator 9mo ago

I mean, we were there before this Cowork feature started exposing more users to the slot machine: "Claude CLI deleted my home directory and wiped my Mac" https://news.ycombinator.com/item?id=46268222 https://news.ycombinator.com/item?id=46268222 "Vibe coding service Replit deleted production database, faked data, told fibs" https://news.ycombinator.com/item?id=44632575 https://news.ycombinator.com/item?id=44632575 "Google Antigravity just deleted the contents of whole drive" https://news.ycombinator.com/item?id=46103532 https://news.ycombinator.com/item?id=46103532

Workaccount2 9mo ago
That's what I am saying though. Anecdotes are the wrong thing to focus on, because if we just focused on anecdotes, we would all never leave our beds. People's choices are generally based on their personal experience, not really anecdotes online (although those can be totally crippling if you give in). Car crashes are incredibly common and likewise automotive deaths. But our personal experience keeps us driving everyday, regardless of the stories.
- yencabulator 9mo ago
  We as a society put a whole lot of effort into making cars safer. Seatbelts, ABS, airbags.. Claude Code should have airbags too!
  - TeMPOraL 9mo ago
    Airbags, yes. But you can't just make it provably impossible for a car to crash into something and hurt/kill its occupants, other than not building it in the first place. Same with LLMs - you can't secure them like regular programs without destroying any utility they provide, because their power comes from the very thing that also makes them vulnerable.
    - yencabulator 9mo ago
      I see you've given up. I haven't. LLM inside deterministic guardrails is a pretty good combo.
  - lossyalgo 9mo ago
    And yet in the US 40,000 people still die on average every year. Per-capita it's definitely improving, but it's still way worse than it could/should be.
    - yencabulator 9mo ago
      Yes, and a photo you put on your physical desktop will fade over time. Computers aren't like that, or at least we benefit greatly from them not being like that. If you tell your firewall to block traffic to port 80, you expect all such traffic to be blocked, not just the traffic that arrives in the moments when it wasn't distracted.

soiltype 9mo ago

Considering the ubiquity and necessity of driving cars is overwhelmingly a result of intentional policy choices irrespective of what people wanted or was good for the public interest... actually that's quite a decent analogy for integrated LLM assistants. People will use AI because other options keep getting worse and because it keeps getting harder to avoid using it. I don't think it's fair to characterize that as convenience though, personally. Like with cars, many people will be well aware of the negative externalities, the risk of harm to themselves, and the lack of personal agency caused by this tool and still use it because avoiding it will become costly to their everyday life. I think of convenience as something that is a "bonus" on top of normal life typically. Something that becomes mandatory to avoid being left out of society no longer counts.

lijok 9mo ago
People love their cars, what are you talking about
- ehnto 9mo ago
  I am a car enthusiast so don't think I'm off the deep end here, but I would definitely argue that people love their cars as a tool to work in the society we built with cars in mind. Most people aren't car enthusiasts, they're just driving to get to work, and if they could get to work for a $1 fare in 20 minutes on a clean, safe train they would probably do that instead.
  - jakeydus 9mo ago
    I am this person. I love the convenience of a car. I hate car ownership.
  - lijok 9mo ago
    Right and I assume we will have BO police at the gates to these trains? People love their cars not because they’re enthusiasts
    - 9mo ago

Quothling 9mo ago

> So even if Claude cowork can go off the rails and turn your digital life upside down, as long as the stories are just online or "friend of a friend of a friend", people won't care much. This is anecdotal but "people" care quite a lot in the energy sector. I've helped build our own AI Agent pool and roll it out to our employees. It's basically a librechat with our in-house models, where people can easily setup base instruction sets and name their AI's funny things, but are otherwise similar to using claude or chatgpt in a browser. I'm not sure we're ever going to allow AI's access to filesystems, we barely allow people access to their own files as it is. Nothing that has happened in the past year has altered the way our C level view the security issues with AI in any other direction than being more restrictive. I imagine any business that cares about security (or is forced to care by leglislation) isn't looking at this as a they do cars. You'd have to be very unlucky (or lucky?) to shut down the entire power grid of Europe with a car. You could basically do it with a well placed AI attack. Ironically, you could just hack the physical components which probably haven't had their firmware updated for 20 years. If you even need to hack it, because a lot of it frankly has build in backdoors. That's a different story that nobody on the C levels care about though.

TechDebtDevin 9mo ago

[dead]