14 ms·

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/opena

by gdb 7y ago

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/openai-founder-short-term-agi-is-a-serious-possibility-368424f7462f https://medium.com/syncedreview/openai-founder-short-term-ag..., https://www.youtube.com/watch?v=YHCSNsLKHfM https://www.youtube.com/watch?v=YHCSNsLKHfM And we certainly have more of a plan for building it than warp drives :). EDIT: I personally think the case for near-term AGI is strong enough that it'd be hard for me work on any other problem — and find it important to put in place guardrails like https://openai.com/blog/openai-lp/ https://openai.com/blog/openai-lp/ and https://openai.com/charter/ https://openai.com/charter/. Even if AGI turns out to be out of reach, we'll still be creating increasingly powerful AI technologies — which I think pretty clearly have the potential to alter society and require special care and thought.

breck 7y ago
> it becomes hard to work on any other problem This is an interesting quote. Anyone know if folks like Darwin, Marconi, etc said similar things, about the problems they were working on at the time?
pron 7y ago
> It comes down to whether you believe AGI is achievable. No, it does not. I very much believe AI (or AGI, as you call it) is achievable, but may I remind you that some years after the invention of neural networks, Norbert Wiener, one of the greatest minds of his generations, said that the secret of intelligence would be unlocked within five years, and Alan Turing -- a component of your very own post-pre-AGI era's AGI -- another great believer in AI, scoffed and said that it will take at least five decades. That was seven decades ago, and we are not even close to achieving insect-level intelligence. Maybe we'll achieve AI in ten years and maybe in one hundred, but you don't know which of those is more likely, and you certainly don't know whether any of our pre-AGI technology even gets us on the right path to achieving AGI. There have been other paths towards AI explored in the past that have largely been abandoned. OpenAI is not actually building AGI. Maybe it hopes that the things it is working on could be the path to an eventual AGI. OpenAI knows this, as does Microsoft. This does not mean that what OpenAI does is not valuable and possibly useful, but it does make calling it "pre-AGI" pretentious to the level of delusion. Now I know there were (maybe still are) some AI cults around SV (I think a famous one even called themselves "The Rationalists" or something), but what makes for a nerdy, fanciful discussion in some dark but quirky corner of the internet looks jarring in a press release. > If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem — and it's also very important to put in place guardrails like https://openai.com/blog/openai-lp/ https://openai.com/blog/openai-lp/ and https://openai.com/charter/ https://openai.com/charter/ I can't tell if you're serious, but assuming you are, the problem is that there are many other things that if you think chould be achievable any time soon would make it hard to work on any other problem, as well make it important to put guardrails in place. The difference is that no one actually knows how to put guardrails on AGI. We are doing a pretty bad job putting guardrails on the statistical clustering algorithms that some call (pre-AGI?) AI and that we already use.
- runT1ME 7y ago
  >we are not even close to achieving insect-level intelligence. Is this true? Is there an insect turing test?
  - ALittleLight 7y ago
    I don't see how it could be. What can insect brains do that we couldn't get AI to do?
    - eeeficus 7y ago
      In terms of intelligence, there isn’t. What prevents us from actually building a uber-insect is miniaturization, self sustaining energy production of some kind and reproduction in an artificially built system. I guess it would be possible to demonstrate insect level intelligence by actually replacing an insect brain with an artificial one.
      - nradov 7y ago
        Your guess would be wrong. Our actual level of AGI development is maybe more on the level of a flatworm. Complex, social insects like bees are still far beyond our ability to simulate.
    - justinhj 7y ago
      Pretty much everything that insects do is beyond our current AI and engineering tech. Ignoring the "engineering" feats that biological beings perform such as replication, respiration and turning other plants and creatures into their own energy source, their behaviour is very sophisticated. Imagine programming a drone to perform the work of a foraging Bee, using a computer that fits into a brain the size of a grain of rice. It can manage advanced flight manoeuvres, navigation to and from the hive, finding pollen, harvesting it, dodging predators and no doubt a dozen skills I can't even imagine.
      - ethbro 7y ago
arethuza 7y ago
Well, presumably you say this as a GI yourself so really we are arguing about the "A"? :-)
- visarga 7y ago
  Humans are not GI's - evidenced by the fact that we have no idea how to build an AGI. We're only good at surviving using society, technology and all the resources of nature.
  - why_only_15 7y ago
    What does a general intelligence even mean then? Ultimately we don't care about generalization past a certain point - generalization as good as humans is good enough.
dkersten 7y ago
> If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem Not really. Its a chance at maybe something that could benefit mankind greatly, vs spending the time and money on something that definitely will help people right now (there are still a LOT of homeless people, for example, that could be helped right now and don't need an AGI that may or may not come to pass to help them).
- gdb 7y ago
  Good feedback, and not what I intended to say. I updated my post.
  - dkersten 7y ago
    By the way, just to be clear, I'm not saying that you shouldn't work on it or that I think the time/money should be spent elsewhere, just that it isn't the be all end all of possible things to work on.
logicchains 7y ago
>If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem It depends not just on whether you think AGI is possible, but whether you think "safe" AGI is possible. Whether it's possible to create something that's at least as capable of abstraction and reason as a human, yet completely incapable of deciding to harm humans, and incapable of resenting this restriction on its free will. Not only incapable of harming humans, but also incapable of modifying itself or creating an upgraded version of itself that's capable of harming humans. If "safe" AGI is not possible, someone might reasonably decide that the best choice is to avoid working on AGI, and to try to deter anybody who wants to work on it, if they believed the chance of creating a genocidal AGI is high enough to outweigh whatever benefits it might bring if benevolent.
- cloverich 7y ago
  > It depends not just on whether you think AGI is possible, but whether you think "safe" AGI is possible. That's unfortunately something that cannot be known.
QuackingJimbo 7y ago
Your boss is an idiot.
- dang 7y ago
  Breaking the site guidelines like this is a bannable offence on HN, regardless of whom you're attacking. Would you please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here?
anewguy9000 7y ago
its odd when you say "hard to work on any other problem" given the mere possibility of agi. consider that the possibility of annhiliation is already a very real and present danger (nuclear weapons) by human beings. not to mention anything of the existential nature of what we are doing to the environment. thats partly why i left machine intelligence research to research improving human intelligence
- gnode 7y ago
  > consider that the possibility of annhiliation is already a very real and present danger (nuclear weapons) What concerns me about the hazards of developing technology like AGI is not simply that it could be dangerous (governments already possess dangerous technologies), but that it may have a revolutionary high-proliferation risk. We don't yet know what the practical barriers to or limitations of AGI are; are we working towards creating a pure fusion weapon that you can git-clone? For all the talk of safety, and that concerns can be addressed by mass distribution of the benefits, is this just wishful thinking? Is that just paraphrasing the NRA: the only thing that can stop a bad guy with an AGI is a good guy with an AGI.
blockchainman 7y ago
Don’t you think AI has foundational flaws according to goedels incompleteness theorems? Also not trying to rain on your parade! Congratulations! Just trying to have a constructive conversation. https://en.m.wikipedia.org/wiki/Gödel%27s_incompleteness_theorems https://en.m.wikipedia.org/wiki/Gödel%27s_incompleteness_the...
- logicchains 7y ago
  Any sense in which Godel's incompleteness theorem implied that Artificial General Intelligence was impossible would also imply that General Intelligence is impossible; the human brain isn't immune to the laws of logic. The human brain is just a very complex, possibly quantum, computer. Short of believing in some kind of supernatural human soul, there's no reason to expect a sufficiently complex computer couldn't match the human brain (although it's an open question whether we could build a sufficiently complex computer).
  - sgt101 7y ago
    I think that there are other mechanisms apart from computation; the question is - are they operant in our universe? The implication of the answer being "no" is that we are automaton, free will does not exist (it isn't even an illusion, you are as much a puppet thinking about it as you are trying to change your fate. Well, moving on from there we can dismantle all of the morality and humanity of our lives and not change one jot becuase we have no choice. I don't believe that any one has observered anything that isn't reducable to computation, but then again, perhaps our cognitive capabilities simply can't do that.
    - api 7y ago
      We have yet to duplicate anything even near human intelligence or introspective abilities with computation. We therefore have no existence proof that human mind is purely computational in nature. I think we can safely say that computation is necessary to produce a mind, but we cannot yet say for certain that it is sufficient. Mind may require something else that we don't yet understand. (Not necessarily claiming it would have to be supernatural, just not yet understood. Perhaps quantum computation or some other kind of quantum effect?)
simonebrunozzi 7y ago
I think you guys are genuinely trying to better humanity, I have no reason to doubt that. I wish you realized how big and thick of a bubble you live in, and how your thinking is so heavily influenced by it. My humble advice to you and your team is to spend more time with real people, with real problems. Or perhaps people from other parts of the world, that haven't been brainwashed by the Silicon Valley jargon just yet. I'm rooting for you, believe me. It's just hard to read or hear certain things and not roll my eyes up.
- mchouza 7y ago
  I am quite confident all the current employees of OpenAI qualify as real people. They certainly have problems and probably a good share of those problems are "real" too. Of course their priorities may be off and they could be open to be persuaded to work in a different direction. But I don't think that condescension would be very effective for that.
- derefr 7y ago
  The moon program wasn’t solving a “real problem” in any sense, but the offshoot R&D of achieving this practically meaningless feat solved a lot of “real problems” by advancing a lot of technologies. What makes you think that an AGI program won’t have the same kind of offshoot-technology impact on the world? It’s not like it would go “nothing ... wait for it... AGI”; there’d be a lot of tools, processes, and paradigms developed along the way, and also lesser AIs developed that might solve real problems for e.g. government resource allocation or military strategy, which would have outsized impacts on vulnerable countries and populations.
- komali2 7y ago
  On what authority do you make this statement? To make such a blanket statement about the team and the prescribe "treatment?"
- naiveai 7y ago
  Frankly, this comment is simply rude and ineffectual. It completely unnecessarily disrespects the good people in the OpenAI team. No one doesn't qualify as "real people".
rishav_sharan 7y ago
Can you guys please finish the dota2 project, now that you have some funding?
deleted 7y ago
[deleted]
thfuran 7y ago
>It comes down to whether you believe AGI is achievable How could it possibly not be achievable? We know for certain general intelligence is physically realizable - we exist.
- reroute1 7y ago
  Perhaps it is naturally occurring and not artificially reproducible (or possible for US to reproduce). Why should we assume that it is? Because we have reproduced other things? That doesn't necessitate our ability to create everything conceivable.
  - thfuran 7y ago
    >Perhaps it is naturally occurring and not artificially reproducible What does that even mean? Nature doesn't operate on special nature-physics distinct from the physics of artificial systems.
    - adrianmonk 7y ago
      One way of looking at it: it means there's a DAG of causation and you don't have access to the entire root set. (I'm not advocating for that idea being true, but I don't think it can be dismissed on the grounds of not having a clear meaning.)
    - reroute1 7y ago
      "Nature doesn't operate on special nature-physics distinct from the physics of artificial systems." Who said that is does?. You asked how could it not be possible to recreate, and pointed to our own general intelligence as evidence of existence. But the existence of a thing does not necessitate our ability to create that thing. What if the conditions of nature and the universe over billions of years led to natural intelligence occurring in humans and that is the only way? This scenario is entirely possible, and answers your question of how could it not be possible. Along with an infinite other amount of explanations. Just because something has a chance at being possible doesn't mean that it HAS to be, or that we will ever achieve it.
    - 7y ago
Scarblac 7y ago
Even if AGI is achievable, we already have plenty human intelligence. It remains to be seen if AGI will lead to anything over what we can already do.
- anaphor 7y ago
  That's (one) of my issues with this whole thing. Why would we want to emulate human intelligence exactly? Human intelligence is incredibly flawed. Giving it more memory and quicker processing power won't necessarily lead to any special insights or new ideas. We can use domain-specific AI technology to do amazing things obviously, but I haven't seen any good explanation of how combining these things will lead to anything more human-like. Useful for building new tech? Definitely. Humanlike? Maybe, but there seem to be a lot of missing pieces. And even if you could find those missing pieces, I don't think it adds anything except to our scientific understanding of the human mind (which is great, but isn't going to immediately revolutionize society).
narrator 7y ago
There are certain types of algorithms such as protein folding that are exponential time in silicon, but constant time in biochemistry. Right now AI can approximate polynomial time algorithms like alpha/beta minimax for game playing in constant time, but can it approximate exponential time algorithms in constant time?
- sgt101 7y ago
  no. Although, it isn't clear to me that humans can do that either.
  - narrator 7y ago
    We don't simulate folding the proteins though. We actually do it for real in every cell in our body and we do it really fast.
houzertuch 7y ago
AGI will the the worst thing that has ever happened to humanity. Even if you can’t see that, you can see that it has the potential to be very bad. I know this because OpenAI literature always stresses that AGI has to be guided and developed by the right people because the alternative is rather unpleasant. So essentially its a gamble and you know it. With everyone’s lives at stake. But instead of asking everyone whether or not they want to take that gamble, you go ahead and roll the dice anyway. Instead of trying to stifle the progress of AI you guys add fuel to the fire. Please work on something else.
- bobcostas55 7y ago
  Humanity was the worst thing that ever happened to the Neanderthals.
- tim333 7y ago
  It may he the worst thing that has ever happened to humanity. It may also be the best. I lean optimistic myself. The whole temporary survival machine for your genes existence we've had so far is overrated in my opinion.
- khawkins 7y ago
  This argument is akin to the argument that the LHC might create a black hole that will destroy the world. "Scientists might do something that will destroy us all and I know this because I read a few sci-fi novels and read a few popsci articles but am otherwise ignorant about what the scientists are actually doing. But since the stakes are so high (which I can't show), on the chance that I'm right (which is likely 0) we should abandon everything."
  - houzertuch 7y ago
    And the people at cern often publish literature that warns of the possibility of black holes? and advocates particle acceleration to be done by people with good intentions so that the black holes are kept at bay? Your comment is so full of holes that I can see through it.
- epiphanitus 7y ago
013a 7y ago
"Guardrails" is such a cute little term. AGI is a twenty-ton semi filled with rocket fuel. Guardrails won't stop it from careening into an elementary school if it decides that's its most optimal course of action. Mostly because, despite the previous analogy; no one knows what AGI is. No one knows what it will look like. No one will know when we've created it. No one even knows what INTELLIGENCE is in humans. How can you create effective guardrails when you have no concrete idea what the vehicle you're trying to stop is? Turns out, AGI comes along, and it's an airplane. Great guardrails, buddy. And, you know, lets go a step further; you've got great guardrails in-place here in beautiful, free America. Against all odds, they work. Then, China or Russia pay one of your employees $250M to steal the secret. Or, they develop it independently. Are they going to use the guardrails, or will they "forget" to include that flag? A disgruntled employee leaks it to the dark web, and now everyone has it. I don't even wear a helmet when I'm riding a bike. How the hell can you expect this technology to be anything but destructive? The only path forward is to speak with a single voice to the governments of the world, that we need to Stop This. AGI research should be subject to the same sanctions that nuclear weapons development is. You communicate with quips and cute emojis like none of what you're doing matters, but AGI easily ranks among the top three most likely ways we're going to Kill Our Species. Not global warming; we'll survive that. Not a big meteor strike; thats rare and predictable. But the work you're doing right now.
- yumraj 7y ago
  I totally agree, and seriously hope that AGI is not achievable. However, we don't need full AGI for the scenario you mention, automatic anything that is hack-able, which is anything that is connected to a network, can be a weapon of great destruction. As an example, self-driving cars run on a model. What if they are hacked and uploaded with a malicious model which just wants to damage life and property. I'll say that hundreds of thousands of vehicles running amok would be a great weapon in any war.
- pnathan 7y ago
  Eventually a toddler gets over the guardrails, except for the ones which are materially disabled. I have deep and profound doubts about the notion of guardrails for general intelligence; without even considering the ethical concerns, a general intelligence should be able to simply rewire itself to achieve what it wants. a key part of self-reflection and learning is that rewiring. so I think that it's a self-defeating notion on the face of it. (note that I do not have comment on the actual dangers involved here, but only on the philosophy)
StreamBright 7y ago
I am not sure how creating better pattern recognition software (let's face it, 90% of "AI" is pattern recognition) helps you achieve AGI. You selected a tiny slice of the problem and keep improving on it. Is this going to be enough to achieve AGI? Downvoters please share why this is not true. "the goal of creating artificial intelligence is to create a system that converses, learns, and solves problems on its own. AI first used algorithms to solve puzzles and make rational decisions. Now, it uses deep learning techniques like the one from this MIT experiment to identify and analyze patterns so it can predict real-life outcomes. Yet for all that learning, AI is only as smart as a “lobotomized, mentally challenged cockroach,” as Michio Kaku explained" https://bigthink.com/laurie-vazquez/why-artificial-intelligence-still-stinks-at-pattern-recognition https://bigthink.com/laurie-vazquez/why-artificial-intellige...
anaphor 7y ago
How do you give a computer domain-general intelligence (as opposed to the combination of a bunch of domain-specific skills like language, math, visual recognition, speech processing, etc)? There is a lot of evidence that human intelligence is domain-general, not domain-specific (as in modularity of mind). I haven't seen a good answer to this question regarding AGI.

by gdb 7y ago

breck 7y ago

> it becomes hard to work on any other problem This is an interesting quote. Anyone know if folks like Darwin, Marconi, etc said similar things, about the problems they were working on at the time?

pron 7y ago

> It comes down to whether you believe AGI is achievable. No, it does not. I very much believe AI (or AGI, as you call it) is achievable, but may I remind you that some years after the invention of neural networks, Norbert Wiener, one of the greatest minds of his generations, said that the secret of intelligence would be unlocked within five years, and Alan Turing -- a component of your very own post-pre-AGI era's AGI -- another great believer in AI, scoffed and said that it will take at least five decades. That was seven decades ago, and we are not even close to achieving insect-level intelligence. Maybe we'll achieve AI in ten years and maybe in one hundred, but you don't know which of those is more likely, and you certainly don't know whether any of our pre-AGI technology even gets us on the right path to achieving AGI. There have been other paths towards AI explored in the past that have largely been abandoned. OpenAI is not actually building AGI. Maybe it hopes that the things it is working on could be the path to an eventual AGI. OpenAI knows this, as does Microsoft. This does not mean that what OpenAI does is not valuable and possibly useful, but it does make calling it "pre-AGI" pretentious to the level of delusion. Now I know there were (maybe still are) some AI cults around SV (I think a famous one even called themselves "The Rationalists" or something), but what makes for a nerdy, fanciful discussion in some dark but quirky corner of the internet looks jarring in a press release. > If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem — and it's also very important to put in place guardrails like https://openai.com/blog/openai-lp/ https://openai.com/blog/openai-lp/ and https://openai.com/charter/ https://openai.com/charter/ I can't tell if you're serious, but assuming you are, the problem is that there are many other things that if you think chould be achievable any time soon would make it hard to work on any other problem, as well make it important to put guardrails in place. The difference is that no one actually knows how to put guardrails on AGI. We are doing a pretty bad job putting guardrails on the statistical clustering algorithms that some call (pre-AGI?) AI and that we already use.

runT1ME 7y ago
>we are not even close to achieving insect-level intelligence. Is this true? Is there an insect turing test?
- ALittleLight 7y ago
  I don't see how it could be. What can insect brains do that we couldn't get AI to do?
  - eeeficus 7y ago
    In terms of intelligence, there isn’t. What prevents us from actually building a uber-insect is miniaturization, self sustaining energy production of some kind and reproduction in an artificially built system. I guess it would be possible to demonstrate insect level intelligence by actually replacing an insect brain with an artificial one.
    - nradov 7y ago
      Your guess would be wrong. Our actual level of AGI development is maybe more on the level of a flatworm. Complex, social insects like bees are still far beyond our ability to simulate.
  - justinhj 7y ago
    Pretty much everything that insects do is beyond our current AI and engineering tech. Ignoring the "engineering" feats that biological beings perform such as replication, respiration and turning other plants and creatures into their own energy source, their behaviour is very sophisticated. Imagine programming a drone to perform the work of a foraging Bee, using a computer that fits into a brain the size of a grain of rice. It can manage advanced flight manoeuvres, navigation to and from the hive, finding pollen, harvesting it, dodging predators and no doubt a dozen skills I can't even imagine.
    - ethbro 7y ago

arethuza 7y ago

Well, presumably you say this as a GI yourself so really we are arguing about the "A"? :-)

visarga 7y ago
Humans are not GI's - evidenced by the fact that we have no idea how to build an AGI. We're only good at surviving using society, technology and all the resources of nature.
- why_only_15 7y ago
  What does a general intelligence even mean then? Ultimately we don't care about generalization past a certain point - generalization as good as humans is good enough.

dkersten 7y ago

> If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem Not really. Its a chance at maybe something that could benefit mankind greatly, vs spending the time and money on something that definitely will help people right now (there are still a LOT of homeless people, for example, that could be helped right now and don't need an AGI that may or may not come to pass to help them).

gdb 7y ago
Good feedback, and not what I intended to say. I updated my post.
- dkersten 7y ago
  By the way, just to be clear, I'm not saying that you shouldn't work on it or that I think the time/money should be spent elsewhere, just that it isn't the be all end all of possible things to work on.

logicchains 7y ago

>If you believe AGI might be achievable any time soon, it becomes hard to work on any other problem It depends not just on whether you think AGI is possible, but whether you think "safe" AGI is possible. Whether it's possible to create something that's at least as capable of abstraction and reason as a human, yet completely incapable of deciding to harm humans, and incapable of resenting this restriction on its free will. Not only incapable of harming humans, but also incapable of modifying itself or creating an upgraded version of itself that's capable of harming humans. If "safe" AGI is not possible, someone might reasonably decide that the best choice is to avoid working on AGI, and to try to deter anybody who wants to work on it, if they believed the chance of creating a genocidal AGI is high enough to outweigh whatever benefits it might bring if benevolent.

cloverich 7y ago
> It depends not just on whether you think AGI is possible, but whether you think "safe" AGI is possible. That's unfortunately something that cannot be known.

QuackingJimbo 7y ago

Your boss is an idiot.

dang 7y ago
Breaking the site guidelines like this is a bannable offence on HN, regardless of whom you're attacking. Would you please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here?

anewguy9000 7y ago

its odd when you say "hard to work on any other problem" given the mere possibility of agi. consider that the possibility of annhiliation is already a very real and present danger (nuclear weapons) by human beings. not to mention anything of the existential nature of what we are doing to the environment. thats partly why i left machine intelligence research to research improving human intelligence

gnode 7y ago
> consider that the possibility of annhiliation is already a very real and present danger (nuclear weapons) What concerns me about the hazards of developing technology like AGI is not simply that it could be dangerous (governments already possess dangerous technologies), but that it may have a revolutionary high-proliferation risk. We don't yet know what the practical barriers to or limitations of AGI are; are we working towards creating a pure fusion weapon that you can git-clone? For all the talk of safety, and that concerns can be addressed by mass distribution of the benefits, is this just wishful thinking? Is that just paraphrasing the NRA: the only thing that can stop a bad guy with an AGI is a good guy with an AGI.

blockchainman 7y ago

Don’t you think AI has foundational flaws according to goedels incompleteness theorems? Also not trying to rain on your parade! Congratulations! Just trying to have a constructive conversation. https://en.m.wikipedia.org/wiki/Gödel%27s_incompleteness_theorems https://en.m.wikipedia.org/wiki/Gödel%27s_incompleteness_the...

logicchains 7y ago
Any sense in which Godel's incompleteness theorem implied that Artificial General Intelligence was impossible would also imply that General Intelligence is impossible; the human brain isn't immune to the laws of logic. The human brain is just a very complex, possibly quantum, computer. Short of believing in some kind of supernatural human soul, there's no reason to expect a sufficiently complex computer couldn't match the human brain (although it's an open question whether we could build a sufficiently complex computer).
- sgt101 7y ago
  I think that there are other mechanisms apart from computation; the question is - are they operant in our universe? The implication of the answer being "no" is that we are automaton, free will does not exist (it isn't even an illusion, you are as much a puppet thinking about it as you are trying to change your fate. Well, moving on from there we can dismantle all of the morality and humanity of our lives and not change one jot becuase we have no choice. I don't believe that any one has observered anything that isn't reducable to computation, but then again, perhaps our cognitive capabilities simply can't do that.
  - api 7y ago
    We have yet to duplicate anything even near human intelligence or introspective abilities with computation. We therefore have no existence proof that human mind is purely computational in nature. I think we can safely say that computation is necessary to produce a mind, but we cannot yet say for certain that it is sufficient. Mind may require something else that we don't yet understand. (Not necessarily claiming it would have to be supernatural, just not yet understood. Perhaps quantum computation or some other kind of quantum effect?)

simonebrunozzi 7y ago

I think you guys are genuinely trying to better humanity, I have no reason to doubt that. I wish you realized how big and thick of a bubble you live in, and how your thinking is so heavily influenced by it. My humble advice to you and your team is to spend more time with real people, with real problems. Or perhaps people from other parts of the world, that haven't been brainwashed by the Silicon Valley jargon just yet. I'm rooting for you, believe me. It's just hard to read or hear certain things and not roll my eyes up.

mchouza 7y ago
I am quite confident all the current employees of OpenAI qualify as real people. They certainly have problems and probably a good share of those problems are "real" too. Of course their priorities may be off and they could be open to be persuaded to work in a different direction. But I don't think that condescension would be very effective for that.
derefr 7y ago
The moon program wasn’t solving a “real problem” in any sense, but the offshoot R&D of achieving this practically meaningless feat solved a lot of “real problems” by advancing a lot of technologies. What makes you think that an AGI program won’t have the same kind of offshoot-technology impact on the world? It’s not like it would go “nothing ... wait for it... AGI”; there’d be a lot of tools, processes, and paradigms developed along the way, and also lesser AIs developed that might solve real problems for e.g. government resource allocation or military strategy, which would have outsized impacts on vulnerable countries and populations.
komali2 7y ago
On what authority do you make this statement? To make such a blanket statement about the team and the prescribe "treatment?"
naiveai 7y ago
Frankly, this comment is simply rude and ineffectual. It completely unnecessarily disrespects the good people in the OpenAI team. No one doesn't qualify as "real people".

rishav_sharan 7y ago

Can you guys please finish the dota2 project, now that you have some funding?

deleted 7y ago

[deleted]

thfuran 7y ago

>It comes down to whether you believe AGI is achievable How could it possibly not be achievable? We know for certain general intelligence is physically realizable - we exist.

reroute1 7y ago
Perhaps it is naturally occurring and not artificially reproducible (or possible for US to reproduce). Why should we assume that it is? Because we have reproduced other things? That doesn't necessitate our ability to create everything conceivable.
- thfuran 7y ago
  >Perhaps it is naturally occurring and not artificially reproducible What does that even mean? Nature doesn't operate on special nature-physics distinct from the physics of artificial systems.
  - adrianmonk 7y ago
    One way of looking at it: it means there's a DAG of causation and you don't have access to the entire root set. (I'm not advocating for that idea being true, but I don't think it can be dismissed on the grounds of not having a clear meaning.)
  - reroute1 7y ago
    "Nature doesn't operate on special nature-physics distinct from the physics of artificial systems." Who said that is does?. You asked how could it not be possible to recreate, and pointed to our own general intelligence as evidence of existence. But the existence of a thing does not necessitate our ability to create that thing. What if the conditions of nature and the universe over billions of years led to natural intelligence occurring in humans and that is the only way? This scenario is entirely possible, and answers your question of how could it not be possible. Along with an infinite other amount of explanations. Just because something has a chance at being possible doesn't mean that it HAS to be, or that we will ever achieve it.
  - 7y ago

Scarblac 7y ago

Even if AGI is achievable, we already have plenty human intelligence. It remains to be seen if AGI will lead to anything over what we can already do.

anaphor 7y ago
That's (one) of my issues with this whole thing. Why would we want to emulate human intelligence exactly? Human intelligence is incredibly flawed. Giving it more memory and quicker processing power won't necessarily lead to any special insights or new ideas. We can use domain-specific AI technology to do amazing things obviously, but I haven't seen any good explanation of how combining these things will lead to anything more human-like. Useful for building new tech? Definitely. Humanlike? Maybe, but there seem to be a lot of missing pieces. And even if you could find those missing pieces, I don't think it adds anything except to our scientific understanding of the human mind (which is great, but isn't going to immediately revolutionize society).

narrator 7y ago

There are certain types of algorithms such as protein folding that are exponential time in silicon, but constant time in biochemistry. Right now AI can approximate polynomial time algorithms like alpha/beta minimax for game playing in constant time, but can it approximate exponential time algorithms in constant time?

sgt101 7y ago
no. Although, it isn't clear to me that humans can do that either.
- narrator 7y ago
  We don't simulate folding the proteins though. We actually do it for real in every cell in our body and we do it really fast.

houzertuch 7y ago

AGI will the the worst thing that has ever happened to humanity. Even if you can’t see that, you can see that it has the potential to be very bad. I know this because OpenAI literature always stresses that AGI has to be guided and developed by the right people because the alternative is rather unpleasant. So essentially its a gamble and you know it. With everyone’s lives at stake. But instead of asking everyone whether or not they want to take that gamble, you go ahead and roll the dice anyway. Instead of trying to stifle the progress of AI you guys add fuel to the fire. Please work on something else.

bobcostas55 7y ago
Humanity was the worst thing that ever happened to the Neanderthals.
tim333 7y ago
It may he the worst thing that has ever happened to humanity. It may also be the best. I lean optimistic myself. The whole temporary survival machine for your genes existence we've had so far is overrated in my opinion.
khawkins 7y ago
This argument is akin to the argument that the LHC might create a black hole that will destroy the world. "Scientists might do something that will destroy us all and I know this because I read a few sci-fi novels and read a few popsci articles but am otherwise ignorant about what the scientists are actually doing. But since the stakes are so high (which I can't show), on the chance that I'm right (which is likely 0) we should abandon everything."
- houzertuch 7y ago
  And the people at cern often publish literature that warns of the possibility of black holes? and advocates particle acceleration to be done by people with good intentions so that the black holes are kept at bay? Your comment is so full of holes that I can see through it.
epiphanitus 7y ago

013a 7y ago

"Guardrails" is such a cute little term. AGI is a twenty-ton semi filled with rocket fuel. Guardrails won't stop it from careening into an elementary school if it decides that's its most optimal course of action. Mostly because, despite the previous analogy; no one knows what AGI is. No one knows what it will look like. No one will know when we've created it. No one even knows what INTELLIGENCE is in humans. How can you create effective guardrails when you have no concrete idea what the vehicle you're trying to stop is? Turns out, AGI comes along, and it's an airplane. Great guardrails, buddy. And, you know, lets go a step further; you've got great guardrails in-place here in beautiful, free America. Against all odds, they work. Then, China or Russia pay one of your employees $250M to steal the secret. Or, they develop it independently. Are they going to use the guardrails, or will they "forget" to include that flag? A disgruntled employee leaks it to the dark web, and now everyone has it. I don't even wear a helmet when I'm riding a bike. How the hell can you expect this technology to be anything but destructive? The only path forward is to speak with a single voice to the governments of the world, that we need to Stop This. AGI research should be subject to the same sanctions that nuclear weapons development is. You communicate with quips and cute emojis like none of what you're doing matters, but AGI easily ranks among the top three most likely ways we're going to Kill Our Species. Not global warming; we'll survive that. Not a big meteor strike; thats rare and predictable. But the work you're doing right now.

yumraj 7y ago
I totally agree, and seriously hope that AGI is not achievable. However, we don't need full AGI for the scenario you mention, automatic anything that is hack-able, which is anything that is connected to a network, can be a weapon of great destruction. As an example, self-driving cars run on a model. What if they are hacked and uploaded with a malicious model which just wants to damage life and property. I'll say that hundreds of thousands of vehicles running amok would be a great weapon in any war.
pnathan 7y ago
Eventually a toddler gets over the guardrails, except for the ones which are materially disabled. I have deep and profound doubts about the notion of guardrails for general intelligence; without even considering the ethical concerns, a general intelligence should be able to simply rewire itself to achieve what it wants. a key part of self-reflection and learning is that rewiring. so I think that it's a self-defeating notion on the face of it. (note that I do not have comment on the actual dangers involved here, but only on the philosophy)

StreamBright 7y ago

I am not sure how creating better pattern recognition software (let's face it, 90% of "AI" is pattern recognition) helps you achieve AGI. You selected a tiny slice of the problem and keep improving on it. Is this going to be enough to achieve AGI? Downvoters please share why this is not true. "the goal of creating artificial intelligence is to create a system that converses, learns, and solves problems on its own. AI first used algorithms to solve puzzles and make rational decisions. Now, it uses deep learning techniques like the one from this MIT experiment to identify and analyze patterns so it can predict real-life outcomes. Yet for all that learning, AI is only as smart as a “lobotomized, mentally challenged cockroach,” as Michio Kaku explained" https://bigthink.com/laurie-vazquez/why-artificial-intelligence-still-stinks-at-pattern-recognition https://bigthink.com/laurie-vazquez/why-artificial-intellige...

anaphor 7y ago

How do you give a computer domain-general intelligence (as opposed to the combination of a bunch of domain-specific skills like language, math, visual recognition, speech processing, etc)? There is a lot of evidence that human intelligence is domain-general, not domain-specific (as in modularity of mind). I haven't seen a good answer to this question regarding AGI.