4 ms·

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissi

by edot 5mo ago

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissions would also blame interns for deleting production whatever. Blame should go up, praise should go down. People always invert these.

giancarlostoro 5mo ago
Yeah, I don't know why anyone would open up a codebase with any prod credentials with an LLM or give prod credentials to an intern / junior developer. I always intentionally had a "PROD" only checkout of my projects so I knew if I was going to try and run it in a PROD mode, that I was going out of my way, there even used to be a VS extension that would change the color of VS completely based on your SLN file path, so I could easily remember which color for VS was for production vs development. I'd have basically a copy that would always be on the latest of the master branch for ease of confirmation.
- ryandrake 5mo ago
  It should take more than "credentials" to even access the prod database, let alone delete it. There's actual customer data there, likely personally identifiable information, maybe their home address, phone number, even real time location? Very sensitive stuff. It should be a Very Big Deal to even access prod. Giving an engineer routine access to prod is a root problem here, along with that engineer laundering that access and giving it to an LLM. At many serious companies, even an insider attempt to access prod could light up a dashboard somewhere, and you might get a call from IT security.
  - giancarlostoro 5mo ago
    Yeah, I'm lucky if I even get READ ONLY credentials for prod in some cases. I don't know why anyone would have all the keys to the prod kingdom.
stingraycharles 5mo ago
> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the fault of AI.
- amluto 5mo ago
  There is a major issue with current AI tools that they want to effectively grant access to everything their user has access to. The whole sandbox structure is wrong (although various people have vibe coded assorted improvements).
  - zahlman 5mo ago
    If you want them to be able to write code and then run tests on that code, it can be a bit difficult to restrict access meaningfully....
    - amluto 5mo ago
      Only for code that can’t be tested in an isolated environment, and designing code that can’t be tested in an isolated environment is generally a mistake for quite a few reasons.
  - yonatan8070 5mo ago
    Another issue I've noticed is they're sometimes very resourceful. For example when Codex can't directly edit file due to sandboxing restrictions, rather than asking "hey can I apply this diff on the file", it'd ask for permission to run a `cat EOF` command to re-write the whole file, which the UI doesn't surface properly (just shows the first line...). This sounds similar to what's described in the "Claude deleted my DB post", it decided "I need to do X", then searched for whatever would let it do X, regardless of intended purpose.
AmbroseBierce 5mo ago
Well, a significant difference is nobody is selling the concept of interns as the end-all solution of humanity's problems, unlike AI.
- program_whiz 5mo ago
  Yeah the usual mott and bailey. Monday -- AI is taking over the world, tremble in fear! Tuesday -- sure it did a boneheaded thing, its just a tool, no better than an intern, actually its _your_ fault, all the data in the entire world isn't enough to train this system not to delete prod!
engeljohnb 5mo ago
Interns are human. Humans can always be held accountable. A computer never can. Therefore, no one should leave a computer in charge of human decisions.
- edot 5mo ago
  Exactly. Thus the blame when an LLM does something dumb should fall on the human who owns the implementation of said LLM. A dead simple example: if I paste confidential information into ChatGPT, that’s on me. If I let Codex have access to an environment where it can get to confidential information, that’s also on me. At best I could also blame my IT department for giving me technical permissions to do such a thing, but still it’s humans at fault (and I believe in taking Extreme Ownership, so I wouldn’t even do that). LLMs are just technology like any other.
  - deleted 5mo ago
    [deleted]
xmcp123 5mo ago
It's a weird world. I also feel pretty confident that if I was an intern who hallucinated regularly at work, I would have been fired, even if I was working for free.
gitaarik 5mo ago
You can however let them work in a sandboxed dev environment where if they break things it doesn't affect the productions system.

by edot 5mo ago

giancarlostoro 5mo ago

Yeah, I don't know why anyone would open up a codebase with any prod credentials with an LLM or give prod credentials to an intern / junior developer. I always intentionally had a "PROD" only checkout of my projects so I knew if I was going to try and run it in a PROD mode, that I was going out of my way, there even used to be a VS extension that would change the color of VS completely based on your SLN file path, so I could easily remember which color for VS was for production vs development. I'd have basically a copy that would always be on the latest of the master branch for ease of confirmation.

ryandrake 5mo ago
It should take more than "credentials" to even access the prod database, let alone delete it. There's actual customer data there, likely personally identifiable information, maybe their home address, phone number, even real time location? Very sensitive stuff. It should be a Very Big Deal to even access prod. Giving an engineer routine access to prod is a root problem here, along with that engineer laundering that access and giving it to an LLM. At many serious companies, even an insider attempt to access prod could light up a dashboard somewhere, and you might get a call from IT security.
- giancarlostoro 5mo ago
  Yeah, I'm lucky if I even get READ ONLY credentials for prod in some cases. I don't know why anyone would have all the keys to the prod kingdom.

stingraycharles 5mo ago

> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the fault of AI.

amluto 5mo ago
There is a major issue with current AI tools that they want to effectively grant access to everything their user has access to. The whole sandbox structure is wrong (although various people have vibe coded assorted improvements).
- zahlman 5mo ago
  If you want them to be able to write code and then run tests on that code, it can be a bit difficult to restrict access meaningfully....
  - amluto 5mo ago
    Only for code that can’t be tested in an isolated environment, and designing code that can’t be tested in an isolated environment is generally a mistake for quite a few reasons.
- yonatan8070 5mo ago
  Another issue I've noticed is they're sometimes very resourceful. For example when Codex can't directly edit file due to sandboxing restrictions, rather than asking "hey can I apply this diff on the file", it'd ask for permission to run a `cat EOF` command to re-write the whole file, which the UI doesn't surface properly (just shows the first line...). This sounds similar to what's described in the "Claude deleted my DB post", it decided "I need to do X", then searched for whatever would let it do X, regardless of intended purpose.

AmbroseBierce 5mo ago

Well, a significant difference is nobody is selling the concept of interns as the end-all solution of humanity's problems, unlike AI.

program_whiz 5mo ago
Yeah the usual mott and bailey. Monday -- AI is taking over the world, tremble in fear! Tuesday -- sure it did a boneheaded thing, its just a tool, no better than an intern, actually its _your_ fault, all the data in the entire world isn't enough to train this system not to delete prod!

engeljohnb 5mo ago

Interns are human. Humans can always be held accountable. A computer never can. Therefore, no one should leave a computer in charge of human decisions.

edot 5mo ago
Exactly. Thus the blame when an LLM does something dumb should fall on the human who owns the implementation of said LLM. A dead simple example: if I paste confidential information into ChatGPT, that’s on me. If I let Codex have access to an environment where it can get to confidential information, that’s also on me. At best I could also blame my IT department for giving me technical permissions to do such a thing, but still it’s humans at fault (and I believe in taking Extreme Ownership, so I wouldn’t even do that). LLMs are just technology like any other.
- deleted 5mo ago
  [deleted]

xmcp123 5mo ago

It's a weird world. I also feel pretty confident that if I was an intern who hallucinated regularly at work, I would have been fired, even if I was working for free.

gitaarik 5mo ago

You can however let them work in a sandboxed dev environment where if they break things it doesn't affect the productions system.