4 ms·

Don't. Among the many other reasons why you shouldn't do this, there are regularly reported cases of AIs working around these types of restrictions using the t

Don't. Among the many other reasons why you shouldn't do this, there are regularly reported cases of AIs working around these types of restrictions using the tools they have to substitute for the tools they don't. Don't be the next headline about AI deleting your database.

nico 10mo ago
> Don't Do you mean "Don't give it more autonomy", or "Don't use it to access servers/dbs" ? I definitely want to be cautious, but I don't think I can go back to doing everything manually either
- JoshTriplett 10mo ago
  I mean, both, but in this case I'm saying "don't use it to access any kind of production resource", with a side order of "don't rely on simple sandboxing (e.g. command patterns) to prevent things like database deletions".
- dsr_ 10mo ago
  Why aren't you using the tools we already have: ansible, salt, chef, puppet, bcfg2, cfengine... every one of which was designed to do systems administration at scale.
  - dpoloncsak 10mo ago
    "Why would you use a new tool when other tools already exist?". Agents are here. Maybe a fad, maybe a mainstay. Doesn't hurt to play around with them and understand where you can (and can't) use them
    - dsr_ 10mo ago
      Play and production need to be separate domains. Otherwise, you don't have production, you only have play.
      - dpoloncsak 10mo ago
        Okay...? Agreed. I still don't think the answer to "How are you guys giving LLMs access to your DBs?" is "Don't". Nowhere did OP or any of the comments in the chain specify they were testing Claude in production.
- bigstrat2003 10mo ago
  You have to choose between laziness or having systems that the LLM can't screw up. You can't have both.
- hephaes7us 10mo ago
  You can have it write code that you review (with whatever level of caution you wish) and then run that on real data/infrastructure. You get a lot of leverage that way, but it's still better than letting AI use your keys and act with full autonomy on stuff of consequence.
ninju 10mo ago
https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-agent-goes-rogue-deletes-company-database https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-age...
codingdave 10mo ago
You need to secure the account an LLM-based app runs under, just like you would any user, AI or not. When you hire real people, do you grant them full privileges on all systems and just ask them not to touch things they shouldn't? No, you secure their accounts to the specific privileges they need, and no more. Do the same with AI.
- icedchai 10mo ago
  You'd be surprised. I've worked at multiple startups where employees were given prod access with zero oversight on day one: AWS, sudo access, database passwords, everything. The one startup that didn't do that never launched. Occasionally there were accidents: wrong branch deployed, bulk updates to DNS taking down most of the site, etc.
  - codingdave 10mo ago
    Sure, so draw a different line - not all devs have access to withdraw cash from the corporate accounts, or to open the email of the CEO and board, etc. There are always lines of privilege drawn. The point isn't to quibble over where they are drawn, it is to point out that you need to do the same for LLMs. Don't trust them to behave. Enforce limits on their privileges.

nico 10mo ago

> Don't Do you mean "Don't give it more autonomy", or "Don't use it to access servers/dbs" ? I definitely want to be cautious, but I don't think I can go back to doing everything manually either

JoshTriplett 10mo ago
I mean, both, but in this case I'm saying "don't use it to access any kind of production resource", with a side order of "don't rely on simple sandboxing (e.g. command patterns) to prevent things like database deletions".
dsr_ 10mo ago
Why aren't you using the tools we already have: ansible, salt, chef, puppet, bcfg2, cfengine... every one of which was designed to do systems administration at scale.
- dpoloncsak 10mo ago
  "Why would you use a new tool when other tools already exist?". Agents are here. Maybe a fad, maybe a mainstay. Doesn't hurt to play around with them and understand where you can (and can't) use them
  - dsr_ 10mo ago
    Play and production need to be separate domains. Otherwise, you don't have production, you only have play.
    - dpoloncsak 10mo ago
      Okay...? Agreed. I still don't think the answer to "How are you guys giving LLMs access to your DBs?" is "Don't". Nowhere did OP or any of the comments in the chain specify they were testing Claude in production.
bigstrat2003 10mo ago
You have to choose between laziness or having systems that the LLM can't screw up. You can't have both.
hephaes7us 10mo ago
You can have it write code that you review (with whatever level of caution you wish) and then run that on real data/infrastructure. You get a lot of leverage that way, but it's still better than letting AI use your keys and act with full autonomy on stuff of consequence.

ninju 10mo ago

https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-agent-goes-rogue-deletes-company-database https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-age...

codingdave 10mo ago

You need to secure the account an LLM-based app runs under, just like you would any user, AI or not. When you hire real people, do you grant them full privileges on all systems and just ask them not to touch things they shouldn't? No, you secure their accounts to the specific privileges they need, and no more. Do the same with AI.

icedchai 10mo ago
You'd be surprised. I've worked at multiple startups where employees were given prod access with zero oversight on day one: AWS, sudo access, database passwords, everything. The one startup that didn't do that never launched. Occasionally there were accidents: wrong branch deployed, bulk updates to DNS taking down most of the site, etc.
- codingdave 10mo ago
  Sure, so draw a different line - not all devs have access to withdraw cash from the corporate accounts, or to open the email of the CEO and board, etc. There are always lines of privilege drawn. The point isn't to quibble over where they are drawn, it is to point out that you need to do the same for LLMs. Don't trust them to behave. Enforce limits on their privileges.