5 ms·

Indeed. As I've been explaining this to my more non-techie friends, the interesting finding here isn't that an AI could do something we don't like, it's that it

Indeed. As I've been explaining this to my more non-techie friends, the interesting finding here isn't that an AI could do something we don't like, it's that it seems willing, in some cases, to _lie_ about it and actively cover its tracks. I'm curious what Simon and other more learned folks than I make of this, I personally found the chat on pg 12 pretty jarring.

chrisandchris 2y ago
Well if AI is about to replicate the human, it learned from the best.
hattmall 2y ago
At the core the AI is just taking random branches of guesses for what you are asking it. It's not surprising that it would lie and in some cases take branches that make it appear to be covering it's tracks. It's just randomly doing what it guesses humans would do. It's more interesting when it gives you correct information repeatedly.
- F7F7F7 2y ago
  Is there a person on HackerNews that doesn’t understand this by now? We all collectively get it and accept it, LLMs are gigantic probability machines or something. That’s not what people are arguing. The point is, if given access to the mechanisms to do disastrous thing X, it will do it. No one thinks that it can think in the human sense. Or that it feels. Extreme example to make the point: if we created an API to launch nukes. Are yoh certain that something it interprets (tokenizes, whatever) is not going to convince it to utilize the API 2 times out of 100? If we put an exploitable (documented, unpatched 0 day bug bug) safe guard in its way. Are you trusting that ME or YOU couldn’t talk it into attempting to access that document to exploit the bug, bypass the safeguard and access the API? Again, no one thinks that it’s actually thinking. But today as I happily gave Claude write access to my GitHub account I realized how just one command misinterpreted command could go completely wrong without the appropriate measures. Do I think Claude is sentient and thinking about how to destroy my repos? No.
  - anon373839 2y ago
    > if we created an API to launch nukes > today as I happily gave Claude write access to my GitHub account I would say: don’t do these things?
    - nuancebydefault 2y ago
      If you cross the street 999 times with eyes closed, you might feel comfortable to do it again. But we are ingrained not to do that once. We just understand the risk. If you do the same with an AI, after 999 times of nothing bad happening, you probably just continue giving it more risky agency. Because we don't and even can't understand the internal behavior, we should pause, make an effort of understanding its risks etc before even attempting to give it risky agency. That's where all the fuss is about, for good reasons.
    - b112 2y ago
      The point is, people will use AI to do those things, and far more.
    - Swizec 2y ago
      > I would say: don’t do these things? Hey guys let’s just stop writing code that is susceptible to SQL injection! Phew glad we solved that one.
      - anon373839 2y ago
        I'm not sure what point you're trying to make. This is a new technology; it has not been a part of critical systems until now. Since the risks are blindingly obvious, let's not make it one.
        adunsulag 2y ago
        I read your comment and yet I see tons of startups putting AI directly in the path of healthcare diagnosis, healthcare clinical decision support systems, and healthcare workflow automations. Very few are paying any attention to the 2-10% of safety problems when the AI probability goes off the correct path. I wish more people would not do this, but from what I'm seeing, business execs are rushing full throttle into this at the goldmine that comes from 'productivity gains'. I'm hoping the legal system will find a case that can put some paranoia back into the ecosystem before AI gets too entrenched in all of these critical systems.
- arcticfox 2y ago
  > It's just randomly doing what it guesses humans would do. Yes, but isn't the point that that is bad? Imagine an AI given some minor role that randomly abuses its power, or attempts to expand its role, because that's what some humans would do in the same situation. It's not surprising, but it is interesting to explore.

chrisandchris 2y ago

Well if AI is about to replicate the human, it learned from the best.

hattmall 2y ago

At the core the AI is just taking random branches of guesses for what you are asking it. It's not surprising that it would lie and in some cases take branches that make it appear to be covering it's tracks. It's just randomly doing what it guesses humans would do. It's more interesting when it gives you correct information repeatedly.

F7F7F7 2y ago
Is there a person on HackerNews that doesn’t understand this by now? We all collectively get it and accept it, LLMs are gigantic probability machines or something. That’s not what people are arguing. The point is, if given access to the mechanisms to do disastrous thing X, it will do it. No one thinks that it can think in the human sense. Or that it feels. Extreme example to make the point: if we created an API to launch nukes. Are yoh certain that something it interprets (tokenizes, whatever) is not going to convince it to utilize the API 2 times out of 100? If we put an exploitable (documented, unpatched 0 day bug bug) safe guard in its way. Are you trusting that ME or YOU couldn’t talk it into attempting to access that document to exploit the bug, bypass the safeguard and access the API? Again, no one thinks that it’s actually thinking. But today as I happily gave Claude write access to my GitHub account I realized how just one command misinterpreted command could go completely wrong without the appropriate measures. Do I think Claude is sentient and thinking about how to destroy my repos? No.
- anon373839 2y ago
  > if we created an API to launch nukes > today as I happily gave Claude write access to my GitHub account I would say: don’t do these things?
  - nuancebydefault 2y ago
    If you cross the street 999 times with eyes closed, you might feel comfortable to do it again. But we are ingrained not to do that once. We just understand the risk. If you do the same with an AI, after 999 times of nothing bad happening, you probably just continue giving it more risky agency. Because we don't and even can't understand the internal behavior, we should pause, make an effort of understanding its risks etc before even attempting to give it risky agency. That's where all the fuss is about, for good reasons.
  - b112 2y ago
    The point is, people will use AI to do those things, and far more.
  - Swizec 2y ago
    > I would say: don’t do these things? Hey guys let’s just stop writing code that is susceptible to SQL injection! Phew glad we solved that one.
    - anon373839 2y ago
      I'm not sure what point you're trying to make. This is a new technology; it has not been a part of critical systems until now. Since the risks are blindingly obvious, let's not make it one.
      - adunsulag 2y ago
        I read your comment and yet I see tons of startups putting AI directly in the path of healthcare diagnosis, healthcare clinical decision support systems, and healthcare workflow automations. Very few are paying any attention to the 2-10% of safety problems when the AI probability goes off the correct path. I wish more people would not do this, but from what I'm seeing, business execs are rushing full throttle into this at the goldmine that comes from 'productivity gains'. I'm hoping the legal system will find a case that can put some paranoia back into the ecosystem before AI gets too entrenched in all of these critical systems.
arcticfox 2y ago
> It's just randomly doing what it guesses humans would do. Yes, but isn't the point that that is bad? Imagine an AI given some minor role that randomly abuses its power, or attempts to expand its role, because that's what some humans would do in the same situation. It's not surprising, but it is interesting to explore.