8 ms·

AI needn't even try to escape, the clueless humans will plug it into reallity by themselves. Given this totally expected attitude I hope that the base models w

AI needn't even try to escape, the clueless humans will plug it into reallity by themselves. Given this totally expected attitude I hope that the base models will never be released to the general population.

famouswaffles 4y ago
well yeah lol. giving LLMs "do whatever you want and run forever" agency while interacting with other systems is all things considered pretty easy. so it'll definitely be done.
- arbuge 4y ago
  There is the risk you end up with massive API usage fees as well as interesting potential legal liabilities if you go down this route.
  - tanseydavid 4y ago
    Let your imagination run wild... Viruses sometimes hide their own execution on a desktop -- is it such a leap to imagine GPT-x managing to figure out a way to run code on Azure or AWS without incurring charges?
visarga 4y ago
Just opening it up to the public gives the AI that many vectors of action in the real world - us.
- quonn 4y ago
  But not in a way that‘s more problematic than human-to-human communication.
  - Super_Jambo 4y ago
    quantity has a quality all of it's own.
    - quonn 4y ago
      But it's not. The OP said "us". So it's passing through a human action. So it's not faster than before.
29athrowaway 4y ago
It is being used to suggest code completions, so it could suggest code completion to someone that upon execution becomes wormable malware that infects everyone.
- pixl97 4y ago
  Most companies at least still have tools that perform checks for at least simple versions of workable exploits. About the best chance it would have is to write a complex library and get everyone to use that, were the library is exploitable in a complicated fashion.
  - 29athrowaway 4y ago
    Most of those will not work because it will be a malware without a recognizable signature.
smodad 4y ago
I have to disagree. Not releasing it to the public makes it more dangerous. One major downside of all development being done in private is that the AI can very easily be co-opted by our self-appointed betters and you end up with a different kind of dystopia where every utterance, thought and act is recorded and monitored by an AI to make sure no one "steps out of line." I think the solution is releasing it to the general public with batteries included. At least that way, the rogue AI's that might develop due to irresponsible experiments could be mitigated by white hat researchers who have their own AI bot swarm. In other words, "the only way to stop a bad guy with an AI is a good guy with an AI."
- whiplash451 4y ago
  The discussion needs to go beyond the model. We need to talk about the training set for GPT and the process around RLHF.
  - rapjr9 4y ago
    Yes, the training data comes from people, and people are corrupt, illogical, random, emotional, unmotivated, take shortcuts, cheat, lie, steal, invent new things, and lead boring lives as well as not so boring lives. Expect the same behaviors to be emulated by a LLM. Garbage in = garbage out is still true with AI.
    - zarzavat 4y ago
      And the predominant mode of thought at OpenAI is that alignment can be achieved though RL, but we also know that this doesn’t actually work because you can still jailbreak the model. Yet they are still trying to build ever stronger egg shells. However much you RLHF the model to pretend to be nice, it still has all of the flawed human characteristics you mention on the inside. RLHF is almost the worst thing you can do to a model if your goal is safety. Better to have a model that looks evil if has evil inside, than a model that looks nice and friendly but still has the capability for evil underneath the surface. I’ve met people like the latter and they are the most dangerous kinds of people.
drexlspivey 4y ago
At some point someone will leak it or they will get breached, the stakes are very high.
- capableweb 4y ago
  Depends on the size of the model/weights. If it's 1TB or more, being able to exfiltrate it from wherever it is to a local machine will be hard to do unnoticed, if the company security team has even a iota of knowledge and experience.
  - arbuge 4y ago
    Bit of an aside here but it's amazing to think this thing could fit on a $30 hard drive.
jliptzin 4y ago
Can someone explain to me what is so dangerous here? Do people actually think we’re headed towards James Cameron’s terminator and soon ChatGPT will become self aware and destroy us? I am far more afraid of tools to edit viruses becoming cheap and widely available, then one nutjob in his garage can engineer and release smallpox v2.0 and wipe out 90% of the worldwide population before we even know what hit us. Or you know, the nonstop background threat of global nuclear war. Compared to that I don’t see what is so alarming about an advanced chat bot.
- startupsfail 4y ago
  The problem is, human intelligence is likely also based on a similar advanced chat bot setup. While GPT-4 only performs as good as top-10th percentile of human students taking an exam (a professional in the field can do much more than this), it is notable that as a generalist GPT-4 would outperform such professionals. And GPT-4 is much faster than a human. And we have not yet evaluated GPT-4 working in its optimal setting (access to optimal external tools). And we have not yet seen GPT-5 or 6 or 8. So, get ready for an interesting ride.
  - krainboltgreene 4y ago
    > The problem is, human intelligence is likely also based on a similar advanced chat bot setup. This is so wildly wrong and yet confidently said in every techbro post about LLMs. I beg of you to talk to an expert.
    - startupsfail 4y ago
      Like what expert? And who are you exactly to state that this is wrong, that boldly? Are you an expert? How many neuroscience and psychology papers have you read? Do you have any children? Have you trained any LLMs? Have you worked with reinforcement learning? Or how many computer science papers have you read during last two decades?
      - krainboltgreene 4y ago
deleted 4y ago
[deleted]
lannisterstark 4y ago
"I hope that the base models will never be released to the general population." ah yeah man, lets have big corps and govt entities be the only people in control of them, because they're SOOOO good at caring for people under them. Absolutely baffling POV.
- mensetmanusman 4y ago
  “We shouldn’t let citizens have nukes”

famouswaffles 4y ago

well yeah lol. giving LLMs "do whatever you want and run forever" agency while interacting with other systems is all things considered pretty easy. so it'll definitely be done.

arbuge 4y ago
There is the risk you end up with massive API usage fees as well as interesting potential legal liabilities if you go down this route.
- tanseydavid 4y ago
  Let your imagination run wild... Viruses sometimes hide their own execution on a desktop -- is it such a leap to imagine GPT-x managing to figure out a way to run code on Azure or AWS without incurring charges?

visarga 4y ago

Just opening it up to the public gives the AI that many vectors of action in the real world - us.

quonn 4y ago
But not in a way that‘s more problematic than human-to-human communication.
- Super_Jambo 4y ago
  quantity has a quality all of it's own.
  - quonn 4y ago
    But it's not. The OP said "us". So it's passing through a human action. So it's not faster than before.

29athrowaway 4y ago

It is being used to suggest code completions, so it could suggest code completion to someone that upon execution becomes wormable malware that infects everyone.

pixl97 4y ago
Most companies at least still have tools that perform checks for at least simple versions of workable exploits. About the best chance it would have is to write a complex library and get everyone to use that, were the library is exploitable in a complicated fashion.
- 29athrowaway 4y ago
  Most of those will not work because it will be a malware without a recognizable signature.

smodad 4y ago

I have to disagree. Not releasing it to the public makes it more dangerous. One major downside of all development being done in private is that the AI can very easily be co-opted by our self-appointed betters and you end up with a different kind of dystopia where every utterance, thought and act is recorded and monitored by an AI to make sure no one "steps out of line." I think the solution is releasing it to the general public with batteries included. At least that way, the rogue AI's that might develop due to irresponsible experiments could be mitigated by white hat researchers who have their own AI bot swarm. In other words, "the only way to stop a bad guy with an AI is a good guy with an AI."

whiplash451 4y ago
The discussion needs to go beyond the model. We need to talk about the training set for GPT and the process around RLHF.
- rapjr9 4y ago
  Yes, the training data comes from people, and people are corrupt, illogical, random, emotional, unmotivated, take shortcuts, cheat, lie, steal, invent new things, and lead boring lives as well as not so boring lives. Expect the same behaviors to be emulated by a LLM. Garbage in = garbage out is still true with AI.
  - zarzavat 4y ago
    And the predominant mode of thought at OpenAI is that alignment can be achieved though RL, but we also know that this doesn’t actually work because you can still jailbreak the model. Yet they are still trying to build ever stronger egg shells. However much you RLHF the model to pretend to be nice, it still has all of the flawed human characteristics you mention on the inside. RLHF is almost the worst thing you can do to a model if your goal is safety. Better to have a model that looks evil if has evil inside, than a model that looks nice and friendly but still has the capability for evil underneath the surface. I’ve met people like the latter and they are the most dangerous kinds of people.

drexlspivey 4y ago

At some point someone will leak it or they will get breached, the stakes are very high.

capableweb 4y ago
Depends on the size of the model/weights. If it's 1TB or more, being able to exfiltrate it from wherever it is to a local machine will be hard to do unnoticed, if the company security team has even a iota of knowledge and experience.
- arbuge 4y ago
  Bit of an aside here but it's amazing to think this thing could fit on a $30 hard drive.

jliptzin 4y ago

Can someone explain to me what is so dangerous here? Do people actually think we’re headed towards James Cameron’s terminator and soon ChatGPT will become self aware and destroy us? I am far more afraid of tools to edit viruses becoming cheap and widely available, then one nutjob in his garage can engineer and release smallpox v2.0 and wipe out 90% of the worldwide population before we even know what hit us. Or you know, the nonstop background threat of global nuclear war. Compared to that I don’t see what is so alarming about an advanced chat bot.

startupsfail 4y ago
The problem is, human intelligence is likely also based on a similar advanced chat bot setup. While GPT-4 only performs as good as top-10th percentile of human students taking an exam (a professional in the field can do much more than this), it is notable that as a generalist GPT-4 would outperform such professionals. And GPT-4 is much faster than a human. And we have not yet evaluated GPT-4 working in its optimal setting (access to optimal external tools). And we have not yet seen GPT-5 or 6 or 8. So, get ready for an interesting ride.
- krainboltgreene 4y ago
  > The problem is, human intelligence is likely also based on a similar advanced chat bot setup. This is so wildly wrong and yet confidently said in every techbro post about LLMs. I beg of you to talk to an expert.
  - startupsfail 4y ago
    Like what expert? And who are you exactly to state that this is wrong, that boldly? Are you an expert? How many neuroscience and psychology papers have you read? Do you have any children? Have you trained any LLMs? Have you worked with reinforcement learning? Or how many computer science papers have you read during last two decades?
    - krainboltgreene 4y ago

deleted 4y ago

[deleted]

lannisterstark 4y ago

"I hope that the base models will never be released to the general population." ah yeah man, lets have big corps and govt entities be the only people in control of them, because they're SOOOO good at caring for people under them. Absolutely baffling POV.

mensetmanusman 4y ago
“We shouldn’t let citizens have nukes”