3 ms·
Guy gives non-deterministic software root access, desaster happens. Movie at eleven. Also, it's not a "confession". It's an LLM stringing together some tokens
by juliansark 5mo ago
Guy gives non-deterministic software root access, desaster happens. Movie at eleven.
Also, it's not a "confession". It's an LLM stringing together some tokens that form words trying to make a pleasing-sounding answer. Plus, the first sentence and the context implies that someone gave it a prompt that told it to never guess around but get stuff done. OP branding this as a confession tells you everything you need to know: total and absolute failure of guard rails, but these guard rails can not be expected to be in an LLM.
- CivBase 5mo agoExactly. Prompts are just weights on a graph traversal. They don't guarantee anything. The LLM does not "understand" the prompts and so it cannot fully adhere to them. They only improve the liklihood it will output what you want. Never ever ever give an LLM access to something you can't afford to break. And stop thinking of them like people.
- aeve890 5mo ago>total and absolute failure of guard rails It seems here the guard rails at failure were the llm users right? Whatever guard rails you can think may be useless against the superior human stupidity. Also, what's the LLM use policy at the SD-6?
- pier25 5mo ago> Guy gives non-deterministic software root access, desaster happens. I agree the guy is an idiot for trusting these AI models. OTOH AI companies keep running and marketing their services with zero accountability for mistakes.
- JohnMakin 5mo agoI guess people are finding out the hard way you do sorta need technical people to say, "hey, maybe this isn't a great idea" rather than trusting marketing hype that says technical skills are dead.
- 0xpgm 5mo agoI wonder, how should an AI company be accountable for non-deterministic nature of AI, which is a fundamental property of the said AI? People have been drinking too much hopium they have lost touch with reality. Everyone needs to properly understand these tools before they use them for anything serious.
- pier25 5mo agoAt the very least, when an agent can delete a production database you should get an obvious warning whenever you enable it. Marketing wouldn't like it though.
- flumes_whims_ 5mo agoHe didn't give it root access, it found root access.
- torginus 5mo agoThis feels like what a dog does. It's incredibly hard to train dogs by punishment, because it's very hard to tell if the dog understands what he did wrong and feels genuine remorse, or is just showing submissive signs at your display of dominance.