9 ms·

Eventually open ai?

by DSingularity 7y ago

Eventually open ai?

oaeide 7y ago
Maybe eventually openAI
- apetrov 7y ago
  maybe eventually open maybe ai
jjeaff 7y ago
"maybe" eventually openAI
ru999gol 7y ago
does anyone serious (non-encumbered) actually believe that?
marvin 7y ago
I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a pragmatic safety-first approach here, as well as towards the near-term economical realities of funding a very expensive project to ensure the fastest possible progress. In the early days of OpenAI, my thoughts were that the project had good intentions, but a misguided focus. The last year has changed that, though. They absolutely seem to be on the right track. Very excited to see their progress over the next years.
- nradov 7y ago
  Not to worry. No one is anywhere close to achieving true AGI so any safety concerns are a moot issue. It's akin to worrying about an alien invasion.
  - DuskStar 7y ago
    > No one is anywhere close to achieving true AGI No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were. EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there. 0: https://intelligence.org/2017/10/13/fire-alarm/ https://intelligence.org/2017/10/13/fire-alarm/
    - marvin 7y ago
      The Manhattan Project is a very apt analogy. Even if you believe that AGI is impossible, it should be possible to appreciate that many billions would quickly be invested in its development if somehow a viable pathway to it became clear. Even if just to a few well-connected experts. This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree that it was impossible not to invest in it, and the delay from «likely impossible» to «done» happened too fast for most observers to notice.
      - idlewords 7y ago
        The Manhattan project happened when the entire conceptual road map to fission weapons was understood. This is manifestly not the case with AI, which can be charitably described as "add computers until magic".
        marvin 7y ago
        I didn’t compare OpenAI to the Manhattan Project. I was pointing out that if a small number of people discover a plausible conceptual pathway to AGI, a similar project will happen.
        idlewords 7y ago
        And I'm pointing out that the conceptual breakthroughs that preceded such an engineering sprint happened in the open literature. Wells was writing sci-fi about atomic weapons in 1914. He based it off of a pop-science book written in 1909. We don't have any such understanding, or even a definition, of 'AGI'.
- z3t4 7y ago
  The atomic bomb was based on science theory. A computer can run many programs and do a great many things, but it will never be able to think by itself.
  - throwawaywego 7y ago
    > The atomic bomb was based on science theory. Our study of (automated) intelligence is based on science too. > A computer ... will never be able to think by itself. Turing wrote an entire paper about this (Computing Machinery and Intelligence), where he rephrases your statement (because he finds it to be meaningless) and devises a test to answer it. He also directly attacks your phrasing of "but it will never": > I believe they are mostly founded on the principle of scientific induction. A man has seen thousands of machines in his lifetime. From what he sees of them he draws a number of general conclusions. They are ugly, each is designed for a very limited purpose, when required for a minutely different purpose they are useless, the variety of behaviour of any one of them is very small, etc., etc. Naturally he concludes that these are necessary properties of machines in general. > A better variant of the objection says that a machine can never "take us by surprise." This statement is a more direct challenge and can be met directly. Machines take me by surprise with great frequency. This is largely because I do not do sufficient calculation to decide what to expect them to do, or rather because, although I do a calculation, I do it in a hurried, slipshod fashion, taking risks.
    - ekianjo 7y ago
      > Our study of (automated) intelligence is based on science too. Can you elaborate which part of sciences you are talking about here?
      - throwawaywego 7y ago
        All sciences that collaborate with the field of AI: Cognitive Science, Neuroscience, Systems Theory, Decision Theory, Information Theory, Mathematics, Physics, Biology, ... Any AI curriculum worth its salt includes the many scientific and philosophical views on intelligence. It is not all alchemy, though the field is in a renewal phase (with horribly hyped nomenclature such as "pre-AGI", and the most impressive implementations coming from industry and government, not academia). And eventhough the atom bomb was based on science too, there is this anecdote from Hamming: > Shortly before the first field test (you realize that no small scale experiment can be done—either you have a critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he said, "It is the probability that the test bomb will ignite the whole atmosphere." I decided I would check it myself! The next day when he came for the answers I remarked to him, "The arithmetic was apparently correct but I do not know about the formulas for the capture cross sections for oxygen and nitrogen—after all, there could be no experiments at the needed energy levels." He replied, like a physicist talking to a mathematician, that he wanted me to check the arithmetic not the physics, and left. I said to myself, "What have you done, Hamming, you are involved in risking all of life that is known in the Universe, and you do not know much of an essential part?" I was pacing up and down the corridor when a friend asked me what was bothering me. I told him. His reply was, "Never mind, Hamming, no one will ever blame you."
- hoseja 7y ago
  I wouldn't be so sure the Manhattan Project-type global arms race isn't already happening.

Eventually open ai?

oaeide 7y ago

Maybe eventually openAI

apetrov 7y ago
maybe eventually open maybe ai

jjeaff 7y ago

"maybe" eventually openAI

ru999gol 7y ago

does anyone serious (non-encumbered) actually believe that?

marvin 7y ago

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a pragmatic safety-first approach here, as well as towards the near-term economical realities of funding a very expensive project to ensure the fastest possible progress. In the early days of OpenAI, my thoughts were that the project had good intentions, but a misguided focus. The last year has changed that, though. They absolutely seem to be on the right track. Very excited to see their progress over the next years.

nradov 7y ago
Not to worry. No one is anywhere close to achieving true AGI so any safety concerns are a moot issue. It's akin to worrying about an alien invasion.
- DuskStar 7y ago
  > No one is anywhere close to achieving true AGI No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were. EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there. 0: https://intelligence.org/2017/10/13/fire-alarm/ https://intelligence.org/2017/10/13/fire-alarm/
  - marvin 7y ago
    The Manhattan Project is a very apt analogy. Even if you believe that AGI is impossible, it should be possible to appreciate that many billions would quickly be invested in its development if somehow a viable pathway to it became clear. Even if just to a few well-connected experts. This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree that it was impossible not to invest in it, and the delay from «likely impossible» to «done» happened too fast for most observers to notice.
    - idlewords 7y ago
      The Manhattan project happened when the entire conceptual road map to fission weapons was understood. This is manifestly not the case with AI, which can be charitably described as "add computers until magic".
      - marvin 7y ago
        I didn’t compare OpenAI to the Manhattan Project. I was pointing out that if a small number of people discover a plausible conceptual pathway to AGI, a similar project will happen.
        idlewords 7y ago
        And I'm pointing out that the conceptual breakthroughs that preceded such an engineering sprint happened in the open literature. Wells was writing sci-fi about atomic weapons in 1914. He based it off of a pop-science book written in 1909. We don't have any such understanding, or even a definition, of 'AGI'.
z3t4 7y ago
The atomic bomb was based on science theory. A computer can run many programs and do a great many things, but it will never be able to think by itself.
- throwawaywego 7y ago
  > The atomic bomb was based on science theory. Our study of (automated) intelligence is based on science too. > A computer ... will never be able to think by itself. Turing wrote an entire paper about this (Computing Machinery and Intelligence), where he rephrases your statement (because he finds it to be meaningless) and devises a test to answer it. He also directly attacks your phrasing of "but it will never": > I believe they are mostly founded on the principle of scientific induction. A man has seen thousands of machines in his lifetime. From what he sees of them he draws a number of general conclusions. They are ugly, each is designed for a very limited purpose, when required for a minutely different purpose they are useless, the variety of behaviour of any one of them is very small, etc., etc. Naturally he concludes that these are necessary properties of machines in general. > A better variant of the objection says that a machine can never "take us by surprise." This statement is a more direct challenge and can be met directly. Machines take me by surprise with great frequency. This is largely because I do not do sufficient calculation to decide what to expect them to do, or rather because, although I do a calculation, I do it in a hurried, slipshod fashion, taking risks.
  - ekianjo 7y ago
    > Our study of (automated) intelligence is based on science too. Can you elaborate which part of sciences you are talking about here?
    - throwawaywego 7y ago
      All sciences that collaborate with the field of AI: Cognitive Science, Neuroscience, Systems Theory, Decision Theory, Information Theory, Mathematics, Physics, Biology, ... Any AI curriculum worth its salt includes the many scientific and philosophical views on intelligence. It is not all alchemy, though the field is in a renewal phase (with horribly hyped nomenclature such as "pre-AGI", and the most impressive implementations coming from industry and government, not academia). And eventhough the atom bomb was based on science too, there is this anecdote from Hamming: > Shortly before the first field test (you realize that no small scale experiment can be done—either you have a critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he said, "It is the probability that the test bomb will ignite the whole atmosphere." I decided I would check it myself! The next day when he came for the answers I remarked to him, "The arithmetic was apparently correct but I do not know about the formulas for the capture cross sections for oxygen and nitrogen—after all, there could be no experiments at the needed energy levels." He replied, like a physicist talking to a mathematician, that he wanted me to check the arithmetic not the physics, and left. I said to myself, "What have you done, Hamming, you are involved in risking all of life that is known in the Universe, and you do not know much of an essential part?" I was pacing up and down the corridor when a friend asked me what was bothering me. I told him. His reply was, "Never mind, Hamming, no one will ever blame you."
hoseja 7y ago
I wouldn't be so sure the Manhattan Project-type global arms race isn't already happening.