4 ms·

What is OpenAI is right and the risks are real? They likely already have some glimpses of GPT-5 internally. And GPT-4 is closely resembles AGI already.

anileated 4y ago
That’s the excuse they’ll use when they convince the government to require a special license to buy powerful GPUs.
- rnk 4y ago
  nvidia won't go for that.
  - int_19h 4y ago
    If LLMs get placed within the scope of ITAR, NVidia won't exactly have a choice. But I suspect that they'll also be getting many government / military contracts going forward.
    - mcast 4y ago
      You might be on to something, NVIDIA stock is up 90% since the start of 2023.
thatguy0900 4y ago
Then it really doesn't matter, because in <10 years all the bad actors who are interested will have made it already. Might as well let people know that they can't trust anything anymore upfront rather than pretend.
jazzyjackson 4y ago
the risks that a text simulator without gaurdrails will be able to generate text we don't like? Or that someone will automate cyberattacks, as if the government isn't already doing it? my greatest fear is that there is only one superintelligence, with access controlled by a monopoly of a few san franciscans deciding what moral boundaries AI will have. I couldn't even get Claude+ to talk to me about houses of zodiac because it insisted it's not an astrologer, it's an AI assistant designed to be helpful blah blah blah, tell me what use is this kind of "safety"?
saiya-jin 4y ago
> GPT-4 is closely resembles AGI already Thats a very bold statement and goes against everything I've read on it so far, care to backup such a claim with some facts? Of course each of us has their own bar for such things, but for most its pretty darn high
- kzrdude 4y ago
  OpenAI people put out this paper https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 called Sparks of Artificial General Intelligence: Early experiments with GPT-4 and as the title makes clear, they think it has hints of AGI. I guess that's a good place to start to answer your question. I don't think this is AGI, but the paper is full of examples where GPT-4 works well and does impressive stuff.
  - ftxbro 4y ago
    you can download tex source for that pdf which at one point (idk if it still does) included the comment "WORK IN PROGRESS - DO NOT SHARE" and the commented out title "First Contact with an AGI System" which they ended up toning down for publication lol
  - airstrike 4y ago
    that paper is unreal... section 6 on theory of mind is downright scary
    - kzrdude 4y ago
      It is. Some things I wonder about, it says things like this: > GPT-4 successfully passes the classic Sally-Anne false-belief test from psychology [BCLF85] (which was modernized to avoid the possibility the answer was memorized from the training data) But it's a language model, generalizing text and performing substitutions on it, is what it excels at. "The car is yellow" is "the <noun> is <descriptor>" and it can substitute in other things, so I'm not sure how their modernization really ensure it does not pattern match on learned texts.
notahacker 4y ago
If they're right that GPT-4 is extremely dangerous, then it's an extraordinarily irresponsible of them to release working implementations as a consumer chat app and integrate it into a search engine. If they're right that LLMs on that scale are generally dangerous but theirs is the exception as they've very carefully calibrated it to be safe, it's extraordinarily irresponsible of them to withhold all details of steps that might make it safe...
- startupsfail 4y ago
  GPT-8 is dangerous. GPT-4 is not.

anileated 4y ago

That’s the excuse they’ll use when they convince the government to require a special license to buy powerful GPUs.

rnk 4y ago
nvidia won't go for that.
- int_19h 4y ago
  If LLMs get placed within the scope of ITAR, NVidia won't exactly have a choice. But I suspect that they'll also be getting many government / military contracts going forward.
  - mcast 4y ago
    You might be on to something, NVIDIA stock is up 90% since the start of 2023.

thatguy0900 4y ago

Then it really doesn't matter, because in <10 years all the bad actors who are interested will have made it already. Might as well let people know that they can't trust anything anymore upfront rather than pretend.

jazzyjackson 4y ago

the risks that a text simulator without gaurdrails will be able to generate text we don't like? Or that someone will automate cyberattacks, as if the government isn't already doing it? my greatest fear is that there is only one superintelligence, with access controlled by a monopoly of a few san franciscans deciding what moral boundaries AI will have. I couldn't even get Claude+ to talk to me about houses of zodiac because it insisted it's not an astrologer, it's an AI assistant designed to be helpful blah blah blah, tell me what use is this kind of "safety"?

saiya-jin 4y ago

> GPT-4 is closely resembles AGI already Thats a very bold statement and goes against everything I've read on it so far, care to backup such a claim with some facts? Of course each of us has their own bar for such things, but for most its pretty darn high

kzrdude 4y ago
OpenAI people put out this paper https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 called Sparks of Artificial General Intelligence: Early experiments with GPT-4 and as the title makes clear, they think it has hints of AGI. I guess that's a good place to start to answer your question. I don't think this is AGI, but the paper is full of examples where GPT-4 works well and does impressive stuff.
- ftxbro 4y ago
  you can download tex source for that pdf which at one point (idk if it still does) included the comment "WORK IN PROGRESS - DO NOT SHARE" and the commented out title "First Contact with an AGI System" which they ended up toning down for publication lol
- airstrike 4y ago
  that paper is unreal... section 6 on theory of mind is downright scary
  - kzrdude 4y ago
    It is. Some things I wonder about, it says things like this: > GPT-4 successfully passes the classic Sally-Anne false-belief test from psychology [BCLF85] (which was modernized to avoid the possibility the answer was memorized from the training data) But it's a language model, generalizing text and performing substitutions on it, is what it excels at. "The car is yellow" is "the <noun> is <descriptor>" and it can substitute in other things, so I'm not sure how their modernization really ensure it does not pattern match on learned texts.

notahacker 4y ago

If they're right that GPT-4 is extremely dangerous, then it's an extraordinarily irresponsible of them to release working implementations as a consumer chat app and integrate it into a search engine. If they're right that LLMs on that scale are generally dangerous but theirs is the exception as they've very carefully calibrated it to be safe, it's extraordinarily irresponsible of them to withhold all details of steps that might make it safe...

startupsfail 4y ago
GPT-8 is dangerous. GPT-4 is not.