5 ms·

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this

by juancn 4y ago

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. The genie is out of the bottle, there's not putting it back in.

nopinsight 4y ago
They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot be pleased with the job displacement risks that GPT-4 plus Plugins may cause, not to mention a more powerful model. They have trained a huge number of college graduates and even now there are significant unemployment/underemployment issues among them. ADDED 2: If you think many companies can do it, please identify a single company outside the US/UK/China with the capability to train an equivalent of GPT-3.5 from scratch.
- famouswaffles 4y ago
  If you counted research, open ai didn't have a lead until gpt-4 nevermind a significant one. most of this is scale. their lead is a few months tops.
  - nopinsight 4y ago
    Engineering such a system is a harder challenge than many types of research. Even the mighty Google, the leader in AI research by many metrics, is catching up. Another example is Meta only finishing OPT-175B, a near equivalent of GPT-3, two years after it. —— Added to reply: GPT-4 got much better results on many benchmarks than PaLM, Google's largest published model [1]. PaLM itself is probably quite a bit better than LamDa in several tasks, according to a chart and a couple of tables here: https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311 It's unclear that Google currently has an internal LLM as good as GPT-4. If they do, they are keeping quiet about it, which seems quite unlikely given the repercussions. [1] GPT-4's benchmark results vs PaLM: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
    - famouswaffles 4y ago
      Google was not catching up before gpt-4. That's my point lol. all the sota llms belonged to google via deepmind and google brain/ai right up to the release of gpt-4. chinchilla, flamingo, flan-palm.
      - nopinsight 4y ago
        GPT-4 was finished in the summer of 2022. Several insiders gave interviews saying they were using it and building guardrails for it for the last 6 months or so. OpenAI doesn’t publish as much as Google so we don’t really know how long or in what periods they were ahead. And there’s no organization outside the US/UK/China with the same caliber of AI engineering output as Google.
- credit_guy 4y ago
  > The only other country OpenAI is not a country, it's a company. GPT models on par with GPT-4 can be trained, well, by companies. You don't need nation-state levels of resources.
  - zarzavat 4y ago
    Training a LLM with GPT-4 like capabilities is very hard. Most AI researchers are concentrated in a few countries. At the moment the countries with the vast majority of the expertise are US, UK and China.
    - MacsHeadroom 4y ago
      It's not remotely intellectually challenging to replicate GPT-4. It just takes a lot of GPUs, something plenty of people all around the world have access to. GPT-2 and GPT-3 are the same algorithm based on the same open source library. GPT-4 most likely is as well. You can literally fork the repo and if you have enough VRAM, cuda cores, and time, you will get GPT-4. High Schoolers could do it. Amateurs are already replicating LLaMA, which is more complex than GPT and not even a month old. (it's just smaller = fewer GPUs required)
      - mirker 4y ago
        You didn’t mention how to gather high quality data. OpenAI has never and will never release that.
      - nopinsight 4y ago
        Engineering such a system is a harder challenge than many types of research. Even the mighty Google, the leader in AI research by many metrics, is catching up. Another example is Meta only finishing OPT-175B, a near equivalent of GPT-3, two years after it. —— GPT-4 got much better results on many benchmarks than PaLM, Google's largest published model [1]. PaLM itself is probably quite a bit better than LamDa in several tasks, according to a chart and a couple of tables here: https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311 It's unclear that Google currently has an internal LLM as good as GPT-4. If they do, they are keeping quiet about it, which seems quite unlikely given the repercussions. [1] GPT-4's benchmark results vs PaLM: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
- ren_engineer 4y ago
  >It’s quite clear that OpenAI has a significant lead over everyone else if their lead was significant they wouldn't have admitted to not releasing more info about GPT-4 in their paper due to commercial reasons. What ever secret sauce they have apparently isn't that significant or they wouldn't be afraid to talk about it
  - NLPaep 4y ago
    Nah, it’s just easy to copy if you lay it out. It’s software.
  - nopinsight 4y ago
    Discovering Newtonian's Laws of Motion were much harder than learning them from others. Were Newtonian's Laws of Motion a significant progress?
- mewpmewp2 4y ago
  If OpenAI as a start up was able to do it, certainly full countries can do it if they see the evidence and will invest in it.
ren_engineer 4y ago
this is basically the same logic behind nuclear weapons, and AI could potentially be even more dangerous if it kept advancing at the rate we've seen in the last few years. In theory the massive amount of compute needed to train and run these at scale could be tracked/regulated similarly to how nuclear refinement facilities are your suggestion is that stopping nuclear proliferation shouldn't have even been attempted, despite the fact it actually worked pretty well
- quonn 4y ago
  But that would only buy us 10 years. Eventually that massive amount won‘t seem very massive anymore compared to what will be available in consumer devices.
- alchemist1e9 4y ago
  > In theory the massive amount of compute needed to train and run these at scale could be tracked/regulated similarly to how nuclear refinement facilities are It seems likely there exists a fully distributed training algorithm and a lot of people are thinking about and I suspect a coordinated training network, perhaps with a reward system, can hopefully be created. Lots of GPUs out there and we just need to figure out how to coordinate them better and shard all the training data.
reasonableklout 4y ago
I don't agree at all. It's totally fair for rival "countries or cultures with different values" to coordinate on matters that threaten everyone's survival. There are many examples of international agreements on technologies that fit this description. For example: - International treaties to avoid nuclear proliferation and development, as other commenters have pointed out - National moratoriums on gain-of-function research - Regulations on biotechnology related to human cloning and human gene editing, and industry/academic norms set by conferences such as the Asilomar conference - International treaties on climate change, such as the Montreal Protocol on CFCs and the hole in the ozone layer Even if we assume that international coordination fails, 2 facts give us some breathing room to pause (not stop!) large experiments to at least figure out what we're doing and how to adapt society: 1. American companies are ahead of Chinese companies (I assume that's what you mean by "other country or culture") right now by at least a few years. The rest of the industry is barely catching up to GPT-3 , which came out in 2020. 2. China cannot unilaterally continue the race right now because their compute supply chain critically depends on technologies that the West has monopolies over, like sub-10nm silicon fabrication, advanced GPU technologies, and ultraviolet lithography. We're already using this lever, actually! For example, Nvidia exports of A100 and H100 GPUs are no longer allowed to China at scale.
- mritchie712 4y ago
  it's orders of magnitude easier to regulate and observe large scale nuclear projects then thousands of hackers spread all over globe.
- GartzenDeHaes 4y ago
  Nvidia chips are designed and manufactured in Taiwan, not "the west."
- lannisterstark 4y ago
  "International treaties to avoid nuclear proliferation and development, as other commenters have pointed out" ...you do realize that countries haven't signed that right, and nothing stops them from pulling out either, right?
winternett 4y ago
They need to stop testing in PROD, and they also need to prove that Ai can function without an Internet connection and without any human intervention at all... Ai should not be deployed deeply until it works more flawlessly, but the same people who hyped and foisted Crypto onto everything and everyone (Including vital world banking infrastructure) are at the wheel on Ai marketing now, and that's not good at all. Those things, as well as vital testing as a gate for deployment, aren't being upheld, and that's exactly what makes promises grandiosely destructive and keeps outcomes harmfully and wildly unpredictable. I think in months to come we'll find out that many of the great new Ai products launched will prove themselves to simply be inauthentic-ly scripted fraud-based solutions backed by hidden human intervention because of this environment of unchecked expansion and eager over-deployment.
ChatGTP 4y ago
If you have time, I really think this short segment from an interview with Max Tegmark is worth a watch [1]. This particular clip is about autonomous weapons. It's quite relevant to your argument, interested on your thoughts. [1] https://www.youtube.com/watch?v=RL4j4KPwNGM https://www.youtube.com/watch?v=RL4j4KPwNGM

by juancn 4y ago

nopinsight 4y ago

They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot be pleased with the job displacement risks that GPT-4 plus Plugins may cause, not to mention a more powerful model. They have trained a huge number of college graduates and even now there are significant unemployment/underemployment issues among them. ADDED 2: If you think many companies can do it, please identify a single company outside the US/UK/China with the capability to train an equivalent of GPT-3.5 from scratch.

famouswaffles 4y ago
If you counted research, open ai didn't have a lead until gpt-4 nevermind a significant one. most of this is scale. their lead is a few months tops.
- nopinsight 4y ago
  Engineering such a system is a harder challenge than many types of research. Even the mighty Google, the leader in AI research by many metrics, is catching up. Another example is Meta only finishing OPT-175B, a near equivalent of GPT-3, two years after it. —— Added to reply: GPT-4 got much better results on many benchmarks than PaLM, Google's largest published model [1]. PaLM itself is probably quite a bit better than LamDa in several tasks, according to a chart and a couple of tables here: https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311 It's unclear that Google currently has an internal LLM as good as GPT-4. If they do, they are keeping quiet about it, which seems quite unlikely given the repercussions. [1] GPT-4's benchmark results vs PaLM: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
  - famouswaffles 4y ago
    Google was not catching up before gpt-4. That's my point lol. all the sota llms belonged to google via deepmind and google brain/ai right up to the release of gpt-4. chinchilla, flamingo, flan-palm.
    - nopinsight 4y ago
      GPT-4 was finished in the summer of 2022. Several insiders gave interviews saying they were using it and building guardrails for it for the last 6 months or so. OpenAI doesn’t publish as much as Google so we don’t really know how long or in what periods they were ahead. And there’s no organization outside the US/UK/China with the same caliber of AI engineering output as Google.
credit_guy 4y ago
> The only other country OpenAI is not a country, it's a company. GPT models on par with GPT-4 can be trained, well, by companies. You don't need nation-state levels of resources.
- zarzavat 4y ago
  Training a LLM with GPT-4 like capabilities is very hard. Most AI researchers are concentrated in a few countries. At the moment the countries with the vast majority of the expertise are US, UK and China.
  - MacsHeadroom 4y ago
    It's not remotely intellectually challenging to replicate GPT-4. It just takes a lot of GPUs, something plenty of people all around the world have access to. GPT-2 and GPT-3 are the same algorithm based on the same open source library. GPT-4 most likely is as well. You can literally fork the repo and if you have enough VRAM, cuda cores, and time, you will get GPT-4. High Schoolers could do it. Amateurs are already replicating LLaMA, which is more complex than GPT and not even a month old. (it's just smaller = fewer GPUs required)
    - mirker 4y ago
      You didn’t mention how to gather high quality data. OpenAI has never and will never release that.
    - nopinsight 4y ago
      Engineering such a system is a harder challenge than many types of research. Even the mighty Google, the leader in AI research by many metrics, is catching up. Another example is Meta only finishing OPT-175B, a near equivalent of GPT-3, two years after it. —— GPT-4 got much better results on many benchmarks than PaLM, Google's largest published model [1]. PaLM itself is probably quite a bit better than LamDa in several tasks, according to a chart and a couple of tables here: https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311 It's unclear that Google currently has an internal LLM as good as GPT-4. If they do, they are keeping quiet about it, which seems quite unlikely given the repercussions. [1] GPT-4's benchmark results vs PaLM: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
ren_engineer 4y ago
>It’s quite clear that OpenAI has a significant lead over everyone else if their lead was significant they wouldn't have admitted to not releasing more info about GPT-4 in their paper due to commercial reasons. What ever secret sauce they have apparently isn't that significant or they wouldn't be afraid to talk about it
- NLPaep 4y ago
  Nah, it’s just easy to copy if you lay it out. It’s software.
- nopinsight 4y ago
  Discovering Newtonian's Laws of Motion were much harder than learning them from others. Were Newtonian's Laws of Motion a significant progress?
mewpmewp2 4y ago
If OpenAI as a start up was able to do it, certainly full countries can do it if they see the evidence and will invest in it.

ren_engineer 4y ago

this is basically the same logic behind nuclear weapons, and AI could potentially be even more dangerous if it kept advancing at the rate we've seen in the last few years. In theory the massive amount of compute needed to train and run these at scale could be tracked/regulated similarly to how nuclear refinement facilities are your suggestion is that stopping nuclear proliferation shouldn't have even been attempted, despite the fact it actually worked pretty well

quonn 4y ago
But that would only buy us 10 years. Eventually that massive amount won‘t seem very massive anymore compared to what will be available in consumer devices.
alchemist1e9 4y ago
> In theory the massive amount of compute needed to train and run these at scale could be tracked/regulated similarly to how nuclear refinement facilities are It seems likely there exists a fully distributed training algorithm and a lot of people are thinking about and I suspect a coordinated training network, perhaps with a reward system, can hopefully be created. Lots of GPUs out there and we just need to figure out how to coordinate them better and shard all the training data.

reasonableklout 4y ago

I don't agree at all. It's totally fair for rival "countries or cultures with different values" to coordinate on matters that threaten everyone's survival. There are many examples of international agreements on technologies that fit this description. For example: - International treaties to avoid nuclear proliferation and development, as other commenters have pointed out - National moratoriums on gain-of-function research - Regulations on biotechnology related to human cloning and human gene editing, and industry/academic norms set by conferences such as the Asilomar conference - International treaties on climate change, such as the Montreal Protocol on CFCs and the hole in the ozone layer Even if we assume that international coordination fails, 2 facts give us some breathing room to pause (not stop!) large experiments to at least figure out what we're doing and how to adapt society: 1. American companies are ahead of Chinese companies (I assume that's what you mean by "other country or culture") right now by at least a few years. The rest of the industry is barely catching up to GPT-3 , which came out in 2020. 2. China cannot unilaterally continue the race right now because their compute supply chain critically depends on technologies that the West has monopolies over, like sub-10nm silicon fabrication, advanced GPU technologies, and ultraviolet lithography. We're already using this lever, actually! For example, Nvidia exports of A100 and H100 GPUs are no longer allowed to China at scale.

mritchie712 4y ago
it's orders of magnitude easier to regulate and observe large scale nuclear projects then thousands of hackers spread all over globe.
GartzenDeHaes 4y ago
Nvidia chips are designed and manufactured in Taiwan, not "the west."
lannisterstark 4y ago
"International treaties to avoid nuclear proliferation and development, as other commenters have pointed out" ...you do realize that countries haven't signed that right, and nothing stops them from pulling out either, right?

winternett 4y ago

They need to stop testing in PROD, and they also need to prove that Ai can function without an Internet connection and without any human intervention at all... Ai should not be deployed deeply until it works more flawlessly, but the same people who hyped and foisted Crypto onto everything and everyone (Including vital world banking infrastructure) are at the wheel on Ai marketing now, and that's not good at all. Those things, as well as vital testing as a gate for deployment, aren't being upheld, and that's exactly what makes promises grandiosely destructive and keeps outcomes harmfully and wildly unpredictable. I think in months to come we'll find out that many of the great new Ai products launched will prove themselves to simply be inauthentic-ly scripted fraud-based solutions backed by hidden human intervention because of this environment of unchecked expansion and eager over-deployment.

ChatGTP 4y ago

If you have time, I really think this short segment from an interview with Max Tegmark is worth a watch [1]. This particular clip is about autonomous weapons. It's quite relevant to your argument, interested on your thoughts. [1] https://www.youtube.com/watch?v=RL4j4KPwNGM https://www.youtube.com/watch?v=RL4j4KPwNGM