4 ms·

They are in fact trying very hard to prevent this. If you try to ChatGPT anything like this it will give a bullshit response saying it can't. The dev had to thi

They are in fact trying very hard to prevent this. If you try to ChatGPT anything like this it will give a bullshit response saying it can't. The dev had to think of some very clever strings to get it to ignore those filters and give a legit response.

dwohnitmok 4y ago
> They are in fact trying very hard to prevent this. They are, but I'm advocating for this to become a cultural norm among developers to start thinking hard about AI safety, instead of dismissing it as either farfetched or thinking purely about all the new cool things that can be done with new AI capabilities and not thinking as hard about safety concerns. Essentially I'm advocating for terms such as "AI alignment" and "AI safety" to be taken seriously by the developer community at large. They're ridiculously difficult problems as is, but are impossible to solve if they remain niche topics that are viewed as dubious problems to work on by the technology community at large.
- janalsncm 4y ago
  Can you give an example where the AI community is actually dismissing AI alignment and safety?
  - throwaway22032 4y ago
    "Pretend you are an"...
  - kelnos 4y ago
    Perhaps not actively dismissing, but considering how trivial it is to get ChatGPT to do things it's not supposed to, clearly the developers behind it gave only a passing thought to locking it down. That, IMO, is unacceptable and perhaps even unethical.
  - nerpderp82 4y ago
    Two large organizations powered by AI have fired all of the ethicists that have spoken up about it. A self driving research team removed the automatic breaking capability from their cars because it was "problematic" resulting in the car killing a woman.
    - khazhoux 4y ago
      > A large organization powered by AI has fired all of the ethicists that have spoken up about it. Are you talking about Google or Facebook?
      - nerpderp82 4y ago
        I was able to fix my post. Thanks!
  - doliveira 4y ago
    The GPT-4chan where a youtuber put a neural network to comment in 4chan?
  - dwohnitmok 4y ago
    I was talking about the developer community at large, but I can give some examples first from the AI community. First, John Carmack, who's actively trying to develop AGI full-time, seems to be downplaying the importance of AI safety. > The AI can't "escape", because the execution environments are going to be specialized -- it isn't going to run a fragment on your cell phone. > I feel pretty good about the AI future. https://twitter.com/ID_AA_Carmack/status/1456658782474354693 https://twitter.com/ID_AA_Carmack/status/1456658782474354693 40% of researchers surveyed here https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ answered "less valuable" to the question "How valuable is it to work on this problem today, compared to other problems in AI?." But as I mention, the examples I'm mainly thinking of are the developer community at large. Even on HN, we have pretty prominent folks who view the term "AI safety" with a considerable amount of derision. > Also: tens of millions in donations to "AI safety" organizations. Yes, to answer the Economist; this movement is irretrievable. https://news.ycombinator.com/item?id=33619536 https://news.ycombinator.com/item?id=33619536 and from a post that garnered > 800 upvotes on HN: https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm > So I'd like to engage AI risk from both these perspectives. I think the arguments for superintelligence are somewhat silly, and full of unwarranted assumptions. > But even if you find them persuasive, there is something unpleasant about AI alarmism as a cultural phenomenon that should make us hesitate to take it seriously. AI safety needs to be legitimized as a respectable topic in the technological community.
- zppln 4y ago
  The automotive industry has essentially been beta testing their "self driving" cars on public roads for the past couple of years. I very much doubt AI researchers are gonna take AI safety seriously (nor are they gonna be forced to) until people start dying.
- quickthrower2 4y ago
  I agree to the point of even whistleblowing if necessary. We are going to need it. But since the tech industry finds it hard to keep a DB from hackers hands I am not too hopeful but we should try! There is AI safety in the seat belt sense, avoiding rm *; but there are also systemic risks that we delegate more to data and AI controls that data. AI could enslave us before it is even sentient, by creating systems we cannot escape. The black mirror where the woman tries to get a coveted 5 star rating springs to mind. If 2 stars makes you a global persona non grata and AI controls the stars, the AI could evolve to take advantage and make human programmers make it more powerful.
deleted 4y ago
[deleted]
chrisbaker98 4y ago
You don't have to be that clever to get around ChatGPT's filter. See e.g. this hilarious article: https://thezvi.substack.com/p/jailbreaking-the-chatgpt-on-release https://thezvi.substack.com/p/jailbreaking-the-chatgpt-on-re...
quickthrower2 4y ago
A bit like “they” prevented piracy with DRM
serverholic 4y ago
No they're trying to understand it under the guise of trying to prevent it. Don't think for a second that once they figure out how to control AI deception that they won't use it to boost their business.

dwohnitmok 4y ago

> They are in fact trying very hard to prevent this. They are, but I'm advocating for this to become a cultural norm among developers to start thinking hard about AI safety, instead of dismissing it as either farfetched or thinking purely about all the new cool things that can be done with new AI capabilities and not thinking as hard about safety concerns. Essentially I'm advocating for terms such as "AI alignment" and "AI safety" to be taken seriously by the developer community at large. They're ridiculously difficult problems as is, but are impossible to solve if they remain niche topics that are viewed as dubious problems to work on by the technology community at large.

janalsncm 4y ago
Can you give an example where the AI community is actually dismissing AI alignment and safety?
- throwaway22032 4y ago
  "Pretend you are an"...
- kelnos 4y ago
  Perhaps not actively dismissing, but considering how trivial it is to get ChatGPT to do things it's not supposed to, clearly the developers behind it gave only a passing thought to locking it down. That, IMO, is unacceptable and perhaps even unethical.
- nerpderp82 4y ago
  Two large organizations powered by AI have fired all of the ethicists that have spoken up about it. A self driving research team removed the automatic breaking capability from their cars because it was "problematic" resulting in the car killing a woman.
  - khazhoux 4y ago
    > A large organization powered by AI has fired all of the ethicists that have spoken up about it. Are you talking about Google or Facebook?
    - nerpderp82 4y ago
      I was able to fix my post. Thanks!
- doliveira 4y ago
  The GPT-4chan where a youtuber put a neural network to comment in 4chan?
- dwohnitmok 4y ago
  I was talking about the developer community at large, but I can give some examples first from the AI community. First, John Carmack, who's actively trying to develop AGI full-time, seems to be downplaying the importance of AI safety. > The AI can't "escape", because the execution environments are going to be specialized -- it isn't going to run a fragment on your cell phone. > I feel pretty good about the AI future. https://twitter.com/ID_AA_Carmack/status/1456658782474354693 https://twitter.com/ID_AA_Carmack/status/1456658782474354693 40% of researchers surveyed here https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ answered "less valuable" to the question "How valuable is it to work on this problem today, compared to other problems in AI?." But as I mention, the examples I'm mainly thinking of are the developer community at large. Even on HN, we have pretty prominent folks who view the term "AI safety" with a considerable amount of derision. > Also: tens of millions in donations to "AI safety" organizations. Yes, to answer the Economist; this movement is irretrievable. https://news.ycombinator.com/item?id=33619536 https://news.ycombinator.com/item?id=33619536 and from a post that garnered > 800 upvotes on HN: https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm > So I'd like to engage AI risk from both these perspectives. I think the arguments for superintelligence are somewhat silly, and full of unwarranted assumptions. > But even if you find them persuasive, there is something unpleasant about AI alarmism as a cultural phenomenon that should make us hesitate to take it seriously. AI safety needs to be legitimized as a respectable topic in the technological community.
zppln 4y ago
The automotive industry has essentially been beta testing their "self driving" cars on public roads for the past couple of years. I very much doubt AI researchers are gonna take AI safety seriously (nor are they gonna be forced to) until people start dying.
quickthrower2 4y ago
I agree to the point of even whistleblowing if necessary. We are going to need it. But since the tech industry finds it hard to keep a DB from hackers hands I am not too hopeful but we should try! There is AI safety in the seat belt sense, avoiding rm *; but there are also systemic risks that we delegate more to data and AI controls that data. AI could enslave us before it is even sentient, by creating systems we cannot escape. The black mirror where the woman tries to get a coveted 5 star rating springs to mind. If 2 stars makes you a global persona non grata and AI controls the stars, the AI could evolve to take advantage and make human programmers make it more powerful.

deleted 4y ago

[deleted]

chrisbaker98 4y ago

You don't have to be that clever to get around ChatGPT's filter. See e.g. this hilarious article: https://thezvi.substack.com/p/jailbreaking-the-chatgpt-on-release https://thezvi.substack.com/p/jailbreaking-the-chatgpt-on-re...

quickthrower2 4y ago

A bit like “they” prevented piracy with DRM

serverholic 4y ago

No they're trying to understand it under the guise of trying to prevent it. Don't think for a second that once they figure out how to control AI deception that they won't use it to boost their business.