4 ms·

Let me flip this argument on its head. Consider this: About 5 years ago several key SV people including Sam Altman, Peter Thiel and Elon Musk became suddenly ve

Let me flip this argument on its head. Consider this: About 5 years ago several key SV people including Sam Altman, Peter Thiel and Elon Musk became suddenly very concerned about AI ethics and started OpenAI. What if they, with this insider status, had already seen a GPT-3 like system at Google, Facebook, Baidu or wherever and its capabilities for political and social manipulation so concerned them that they started OpenAI in an effort to bring this tech out of the shadows and into the sunlight so we could debate it and regulate it. GPT-3 might not be a state of the art breakthrough. It could be just catching up with where the big tech companies were 5 years ago so that we can finally see what they are capable of. Corporate secrets are a normal part of doing business and maybe the tech companies didn't like the PR they would have gotten from publicizing something like this. Remember the blowback from Google's project that called business for their store hours? They already struggle with regulators across the world as it is. Do we really believe that little OpenAI is so much farther ahead of Google like the posted article posits?

notadonut 6y ago
You don't need secrets for your theory of insiders to work. What Thiel and Musk saw was DeepMind. Then they invested in it. Altman acted later. And the vision of what AI was becoming was voiced much earlier by Yudkowsky, whose Singularity Institute received funding from Thiel. If anything, they heard what a few prophets were shouting. They saw some early demos in a startup pitch. They responded and DeepMind's work soon became as public as AI research is. That is to say, most people ignored it until the Google acquisition.
- exolymph 6y ago
  Yudkowsky's outfit is the Machine Intelligence Research Institute, not the Singularity Institute. The latter might be Bostrom? Can't recall.
  - pjscott 6y ago
    MIRI was, until 2013, called the Singularity Institute for Artificial Intelligence. Apparently the name change was part of a deal with Singularity University to avoid brand confusion. Announcement here: https://intelligence.org/2013/01/30/we-are-now-the-machine-intelligence-research-institute-miri/ https://intelligence.org/2013/01/30/we-are-now-the-machine-i...
    - notadonut 6y ago
      Thank you. Thiel was an early backer of SI and attendee of the Singularity Summits that ran from 2006-2012. https://en.wikipedia.org/wiki/Singularity_Summit https://en.wikipedia.org/wiki/Singularity_Summit Soon after that, awareness of AI and superintelligence went mainstream, and we got FHI, FLI, etc. I don't know if Thiel backs MIRI as he did SI. Arguably, he doesn't need to. He made his money on DeepMind and helped trigger a larger movement, and other institutions with a lot more resources, like Alphabet and MSFT, carry forward the torch.
      - Nuzzerino 6y ago
        Thiel wasn't an attendee, he was one of the people running the conference
        notadonut 6y ago
        He was attendee and sponsor and running the conference.
stjo 6y ago
Sounds like a nice conspiracy, but realistically, how could they hide something like that? Presumably there are hundreds or more employees working on this. If Elon Musk et al. heard about it 5 years ago, this must be one of the best kept secrets in recent history.
- ColanR 6y ago
  > how could they hide something like that? Very carefully. I mean, that's not much of an argument. Lots of stuff is successfully kept secret. The US managed to keep a lid on their surveilance for decades (iirc) before the lid got blown on that, and people used to give the same argument you are in that context, too. What's the alternative? Do you think megacorps never keep illicit things under wraps for extended periods of time?
  - ben_w 6y ago
    There is a formula for working out how long it will take for a conspiracy theory to be made public based on how many people are involved in it. I guess you could use it in reverse and produce an upper limit on how many people could be involved in a conspiracy if you assume that it has been secret for five years. That said, Elon Musk gives every impression of being a massive chatterbox who can’t keep his mouth shut even when its the SEC threatening to take Tesla away from him, so I very much doubt any conspiracy involves him.
    - riffraff 6y ago
      for those interested, the equation for estimating if a CT would last was published by a Dr. Grimes in 2016[0], it's pretty interesting :) [0] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0147905 https://journals.plos.org/plosone/article?id=10.1371/journal...
      - ColanR 6y ago
        I would love to see a follow up to that. From what I recall, the PRISM program they used in that study wasn't the only one of its kind, and anything from the 80s or earlier wasn't included. They also didn't make mention of any of the CIA's old projects from the 70s / 80s that get declassified every few years. I wonder how much those would skew the results.
grenoire 6y ago
>"bring this tech out of the shadows and into the sunlight so we could debate it and regulate it" Ahh... I'm not so sure; see the comment by blueyes. Had it been OpenAI's goal to engage in debate and regulation for this technology, they would have been vocal about that aspect of their work already.
- dougmwne 6y ago
  That's fair, and looking at some of the founding statements it seems they were much more interested in getting the tech out there and available to everyone equality and didn't seem to be big believers in regulation. I think I am projecting my own belief of the importance of regulation onto this.
Nuzzerino 6y ago
Five years ago? Peter Thiel hosted the Singularity Summit in 2006, and a major premise of that summit was AI Ethics...
claytoneast 6y ago
from Gwern, @ https://www.gwern.net/newsletter/2020/05 https://www.gwern.net/newsletter/2020/05: "This year, GPT-3 is scary because it’s a magnificently obsolete architecture from early 2018, which is small & shallow compared to what’s possible3, with a simple uniform architecture4 trained in the dumbest way possible (unidirectional prediction of next text token) on a single impoverished modality (random Internet HTML text dumps5) on tiny data (fits on a laptop), sampled in a dumb way6, and yet, the first version already manifests crazy runtime meta-learning—and the scaling curves still are not bending!" It's probably not a state-of-the-art breakthrough at this point. Who knows what OpenAI has done in the intervening two years?
rytill 6y ago
Or maybe, just maybe, it was and still is possible with public evidence and independent thinking to understand the importance of AGI and AI ethics.
jimmyvalmer 6y ago
"Flipping this argument on its head" = repurposing a detailed, quantitative cost analysis to argue we're not in overhang. "Not reading OP and spouting a non-quantitative conspiracy theory" = what you did.
- dougmwne 6y ago
  You are absolutely right! I did read TFA, but it is knowledgeable and well considered so I can't address it specifically. It is still fun to challenge one of its assumptions, namely that Google is now playing catch-up. It's standard devil's advocate stuff, but seemed a fine vein to mine. Yes, Google is probably now behind in language generation vs OpenAI, but what if they weren't and how far might that go?
gld773 6y ago
>so we could debate it and regulate it. but you'd have no way of enforcing any treaty so that's a moot point and they would know this. I think making people aware of the importance of the control or value loading problems is a much better use of efforts.
rictic 6y ago
Nick Bostrom's 2014 book Superintelligence is the reason that many big name figures started to take the threat seriously.
dragongod2718 6y ago
u/Gwern's [explanation on why Google didn't produce GPT-3 earlier](https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang?commentId=jbD8siv7GMWxRro43 https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-...): As far as I can tell, this is what is going on: they do not have any such thing, because GB and DM do not believe in the scaling hypothesis the way that Sutskever, Amodei and others at OA do. GB is entirely too practical and short-term focused to dabble in such esoteric & expensive speculation, although Quoc's group occasionally surprises you. They'll dabble in something like GShard, but mostly because they expect to be likely to be able to deploy it or something like it to production in Google Translate. DM (particularly Hassabis, I'm not sure about Legg's current views) believes that AGI will require effectively replicating the human brain module by module, and that while these modules will be extremely large and expensive by contemporary standards, they still need to be invented and finetuned piece by piece, with little risk or surprise until the final assembly. That is how you get DM contraptions like Agent57 which are throwing the kitchen sink at the wall to see what sticks, and why they place such emphasis on neuroscience as inspiration and cross-fertilization. When someone seems to have come up with a scalable architecture for a problem, like AlphaZero or AlphaStar, they are willing to pour on the gas to make it scale, but otherwise, incremental refinement on ALE and then DMLab is the game plan. Because they have locked up so much talent and have so much proprietary code and believe all of that is a major moat to any competitor trying to replicate the complicated brain, they are fairly easygoing. OA, lacking anything like DM's long-term funding from Google or its enormous headcount, is making a startup-like bet that they know the secret: the scaling hypothesis is true and very simple DRL algorithms like PPO on top of large simple architectures like RNNs or Transformers can emerge and meta-learn their way to powerful capabilities, enabling further funding for still more compute & scaling, in a virtuous cycle. And if OA is wrong to trust in the God of Straight Lines On Graphs, well, they never could compete with DM directly using DM's favored approach, and were always going to be an also-ran footnote. While all of this hypothetically can be replicated relatively easily (never underestimate the amount of tweaking and special sauce it takes) by competitors if they wished (the necessary amounts of compute budgets are still trivial in terms of Big Science or other investments like AlphaGo or AlphaStar or Waymo, after all), said competitors are too hidebound and deeply philosophically wrong to ever admit fault and try to overtake OA until it's too late. This might seem absurd, but look at the repeated criticism of OA every time they release a new example of the scaling hypothesis, from GPT-1 to Dactyl to OA5 to GPT-2 to iGPT to GPT-3... (When faced with the choice between having to admit all their fancy hard work is a dead-end, swallow the bitter lesson, and start budgeting tens of millions of compute, or between writing a tweet explaining how, "actually, GPT-3 shows that scaling is a dead end and it's just imitation intelligence" - most people will get busy on the tweet!)

notadonut 6y ago

You don't need secrets for your theory of insiders to work. What Thiel and Musk saw was DeepMind. Then they invested in it. Altman acted later. And the vision of what AI was becoming was voiced much earlier by Yudkowsky, whose Singularity Institute received funding from Thiel. If anything, they heard what a few prophets were shouting. They saw some early demos in a startup pitch. They responded and DeepMind's work soon became as public as AI research is. That is to say, most people ignored it until the Google acquisition.

exolymph 6y ago
Yudkowsky's outfit is the Machine Intelligence Research Institute, not the Singularity Institute. The latter might be Bostrom? Can't recall.
- pjscott 6y ago
  MIRI was, until 2013, called the Singularity Institute for Artificial Intelligence. Apparently the name change was part of a deal with Singularity University to avoid brand confusion. Announcement here: https://intelligence.org/2013/01/30/we-are-now-the-machine-intelligence-research-institute-miri/ https://intelligence.org/2013/01/30/we-are-now-the-machine-i...
  - notadonut 6y ago
    Thank you. Thiel was an early backer of SI and attendee of the Singularity Summits that ran from 2006-2012. https://en.wikipedia.org/wiki/Singularity_Summit https://en.wikipedia.org/wiki/Singularity_Summit Soon after that, awareness of AI and superintelligence went mainstream, and we got FHI, FLI, etc. I don't know if Thiel backs MIRI as he did SI. Arguably, he doesn't need to. He made his money on DeepMind and helped trigger a larger movement, and other institutions with a lot more resources, like Alphabet and MSFT, carry forward the torch.
    - Nuzzerino 6y ago
      Thiel wasn't an attendee, he was one of the people running the conference
      - notadonut 6y ago
        He was attendee and sponsor and running the conference.

stjo 6y ago

Sounds like a nice conspiracy, but realistically, how could they hide something like that? Presumably there are hundreds or more employees working on this. If Elon Musk et al. heard about it 5 years ago, this must be one of the best kept secrets in recent history.

ColanR 6y ago
> how could they hide something like that? Very carefully. I mean, that's not much of an argument. Lots of stuff is successfully kept secret. The US managed to keep a lid on their surveilance for decades (iirc) before the lid got blown on that, and people used to give the same argument you are in that context, too. What's the alternative? Do you think megacorps never keep illicit things under wraps for extended periods of time?
- ben_w 6y ago
  There is a formula for working out how long it will take for a conspiracy theory to be made public based on how many people are involved in it. I guess you could use it in reverse and produce an upper limit on how many people could be involved in a conspiracy if you assume that it has been secret for five years. That said, Elon Musk gives every impression of being a massive chatterbox who can’t keep his mouth shut even when its the SEC threatening to take Tesla away from him, so I very much doubt any conspiracy involves him.
  - riffraff 6y ago
    for those interested, the equation for estimating if a CT would last was published by a Dr. Grimes in 2016[0], it's pretty interesting :) [0] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0147905 https://journals.plos.org/plosone/article?id=10.1371/journal...
    - ColanR 6y ago
      I would love to see a follow up to that. From what I recall, the PRISM program they used in that study wasn't the only one of its kind, and anything from the 80s or earlier wasn't included. They also didn't make mention of any of the CIA's old projects from the 70s / 80s that get declassified every few years. I wonder how much those would skew the results.

grenoire 6y ago

>"bring this tech out of the shadows and into the sunlight so we could debate it and regulate it" Ahh... I'm not so sure; see the comment by blueyes. Had it been OpenAI's goal to engage in debate and regulation for this technology, they would have been vocal about that aspect of their work already.

dougmwne 6y ago
That's fair, and looking at some of the founding statements it seems they were much more interested in getting the tech out there and available to everyone equality and didn't seem to be big believers in regulation. I think I am projecting my own belief of the importance of regulation onto this.

Nuzzerino 6y ago

Five years ago? Peter Thiel hosted the Singularity Summit in 2006, and a major premise of that summit was AI Ethics...

claytoneast 6y ago

from Gwern, @ https://www.gwern.net/newsletter/2020/05 https://www.gwern.net/newsletter/2020/05: "This year, GPT-3 is scary because it’s a magnificently obsolete architecture from early 2018, which is small & shallow compared to what’s possible3, with a simple uniform architecture4 trained in the dumbest way possible (unidirectional prediction of next text token) on a single impoverished modality (random Internet HTML text dumps5) on tiny data (fits on a laptop), sampled in a dumb way6, and yet, the first version already manifests crazy runtime meta-learning—and the scaling curves still are not bending!" It's probably not a state-of-the-art breakthrough at this point. Who knows what OpenAI has done in the intervening two years?

rytill 6y ago

Or maybe, just maybe, it was and still is possible with public evidence and independent thinking to understand the importance of AGI and AI ethics.

jimmyvalmer 6y ago

"Flipping this argument on its head" = repurposing a detailed, quantitative cost analysis to argue we're not in overhang. "Not reading OP and spouting a non-quantitative conspiracy theory" = what you did.

dougmwne 6y ago
You are absolutely right! I did read TFA, but it is knowledgeable and well considered so I can't address it specifically. It is still fun to challenge one of its assumptions, namely that Google is now playing catch-up. It's standard devil's advocate stuff, but seemed a fine vein to mine. Yes, Google is probably now behind in language generation vs OpenAI, but what if they weren't and how far might that go?

gld773 6y ago

>so we could debate it and regulate it. but you'd have no way of enforcing any treaty so that's a moot point and they would know this. I think making people aware of the importance of the control or value loading problems is a much better use of efforts.

rictic 6y ago

Nick Bostrom's 2014 book Superintelligence is the reason that many big name figures started to take the threat seriously.

dragongod2718 6y ago

u/Gwern's [explanation on why Google didn't produce GPT-3 earlier](https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang?commentId=jbD8siv7GMWxRro43 https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-...): As far as I can tell, this is what is going on: they do not have any such thing, because GB and DM do not believe in the scaling hypothesis the way that Sutskever, Amodei and others at OA do. GB is entirely too practical and short-term focused to dabble in such esoteric & expensive speculation, although Quoc's group occasionally surprises you. They'll dabble in something like GShard, but mostly because they expect to be likely to be able to deploy it or something like it to production in Google Translate. DM (particularly Hassabis, I'm not sure about Legg's current views) believes that AGI will require effectively replicating the human brain module by module, and that while these modules will be extremely large and expensive by contemporary standards, they still need to be invented and finetuned piece by piece, with little risk or surprise until the final assembly. That is how you get DM contraptions like Agent57 which are throwing the kitchen sink at the wall to see what sticks, and why they place such emphasis on neuroscience as inspiration and cross-fertilization. When someone seems to have come up with a scalable architecture for a problem, like AlphaZero or AlphaStar, they are willing to pour on the gas to make it scale, but otherwise, incremental refinement on ALE and then DMLab is the game plan. Because they have locked up so much talent and have so much proprietary code and believe all of that is a major moat to any competitor trying to replicate the complicated brain, they are fairly easygoing. OA, lacking anything like DM's long-term funding from Google or its enormous headcount, is making a startup-like bet that they know the secret: the scaling hypothesis is true and very simple DRL algorithms like PPO on top of large simple architectures like RNNs or Transformers can emerge and meta-learn their way to powerful capabilities, enabling further funding for still more compute & scaling, in a virtuous cycle. And if OA is wrong to trust in the God of Straight Lines On Graphs, well, they never could compete with DM directly using DM's favored approach, and were always going to be an also-ran footnote. While all of this hypothetically can be replicated relatively easily (never underestimate the amount of tweaking and special sauce it takes) by competitors if they wished (the necessary amounts of compute budgets are still trivial in terms of Big Science or other investments like AlphaGo or AlphaStar or Waymo, after all), said competitors are too hidebound and deeply philosophically wrong to ever admit fault and try to overtake OA until it's too late. This might seem absurd, but look at the repeated criticism of OA every time they release a new example of the scaling hypothesis, from GPT-1 to Dactyl to OA5 to GPT-2 to iGPT to GPT-3... (When faced with the choice between having to admit all their fancy hard work is a dead-end, swallow the bitter lesson, and start budgeting tens of millions of compute, or between writing a tweet explaining how, "actually, GPT-3 shows that scaling is a dead end and it's just imitation intelligence" - most people will get busy on the tweet!)