6 ms·

AMA: the OpenAI research team

130 points by sushirain 11y ago

meeper16 11y ago
Any chance at asking questions here? Whoops, guess I just did...
- gdb 11y ago
  waves
  - jk4930 11y ago
    Do you have info materials about OpenAI I could use in AI events? Will you work with 3rd parties to rise awareness and support, recruit talent and external collaborators (e.g. research groups)?
    - gdb 11y ago
      No particular materials right now besides the website. We expect to collaborate with many third parties over time. We're just getting started, so not sure on the specifics yet!
      - jk4930 11y ago
        Okay. May I write you an email to establish the connection? I intended to contact OpenAI later this month, but now that we're talking...
        gdb 11y ago
        Feel free to get in touch: gdb@openai.com!
        jk4930 11y ago
        I'll send you a mail tomorrow. Thanks.
    - nl 11y ago
      May I suggest Kaggle? The AllenAI Institute is currently running a pretty interesting contest there at the moment, with pretty direct links to OpenAI's mission. Of course Kaggle often degenerates into a "who can drive xgboost[1] the best", but some of the feature engineering is quite novel. A decent Winograd schema challenge would be pretty interesting. [1] https://github.com/dmlc/xgboost#whats-new https://github.com/dmlc/xgboost#whats-new
  - emcq 11y ago
    Do you believe there is a limit to what models we can train with layers of differentiable units?
TeMPOraL 11y ago
Scott Alexander expressed some important concerns about the project[0]. Edited: <del>So far they've managed to label them as "coming from the LessWrong background" and subsequently dismiss via appeal to a strawman Paperclip Maximizer. It doesn't give me much confidence in them.</del> <ins>Nevermind. I didn't realize this comment was not made by an OpenAI representative. Also, we could use a strikethrough formatting tag on HN. 'dang?</ins> I hope they eventually address those points though. [0] - http://slatestarcodex.com/2015/12/17/should-ai-be-open/ http://slatestarcodex.com/2015/12/17/should-ai-be-open/
- gdb 11y ago
  To be clear, the following comment is what we wrote on the subject: https://www.reddit.com/r/MachineLearning/comments/404r9m/ama_the_openai_research_team/cysejas https://www.reddit.com/r/MachineLearning/comments/404r9m/ama.... The subsequent replies are not affiliated with OpenAI.
  - TeMPOraL 11y ago
    Thanks, I missed that. I didn't notice the OpenAI flair. I apologize for mistakenly assigning this comment to your group.
- gjm11 11y ago
  The person who referred to "LessWrong background" and talked about paperclip maximizers wasn't (at least, so it looks to me) mocking or dismissing. The last paragraph of that comment does say "the LessWrong folks tend to be overly dramatic in their concerns" but goes on immediately to add "But they do have a point that the problem of controlling something much more intelligent than yourself is hard [...] and, if truly super-human intelligence is practically possible, then it needs to be solved before we build it".
AndrewKemendo 11y ago
I missed the AMA, but glad to see it here and it looks like there are some team members monitoring this thread so I'll throw some questions out there. I did my MS in AGI at the National Intelligence University with Ben Goertzel as my outside research advisor. My thesis was to determine what were defense implications of an AGI and who was making the best progress toward actually building one. Since then the DoD has started to take an interest in AGI, and in fact today, during my one weekend a month drill at the Pentagon, I had a great long conversation with Maj. General Seng [1] who is heading up efforts around implementing ISR systems with more autonomous capabilities and exploring how an AGI would be utilized in defense. One of our big open questions is what's the "stack" for AGI, conceptually? I didn't come to any conclusion on this and had to make a lot of assumptions to close out the research. I would be curious to hear the OpenAI team's thoughts on it. Will you all be coming to AGI 16 this year in New York? [1] http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Article/108644/major-general-jocelyn-m-seng.aspx http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Arti...
- eli_gottlieb 11y ago
  >Since then the DoD has started to take an interest in AGI, and in fact today, during my one weekend a month drill at the Pentagon, I had a great long conversation with Maj. General Seng [1] who is heading up efforts around implementing ISR systems with more autonomous capabilities and exploring how an AGI would be utilized in defense. Isn't that basically the most obviously bad idea ever, so terribly stupid that there have been several movies chronicling how massively bad an idea it actually is to build "AGI" for military goals?
  - AndrewKemendo 11y ago
    I think you have been watching too many movies.
    - TeMPOraL 11y ago
      Movies sometimes get this thing right though. Don't forget that Dr. Strangelove pretty much turned out to be a documentary!
chegra 11y ago
1. Do you believe solving AI will come from a big company where the employees solving it would have little ownership of the company? 2. Currently, there is no solid test for Artificial General Intelligence. How much of a priority is it to create one? ------------------------------------------------------- Answering my own questions: 1. No, the incentive is just not there. 2. Currently, this is my main goal related to Artificial General Intelligence. One should know how far or near they are from creating Artificial General Intelligence.
- AndrewKemendo 11y ago
  Currently, there is no solid test for Artificial General Intelligence. How much of a priority is it to create one? There is of course no commonly agreed upon one, but in my research but the best one I found was the Universal Anytime Intelligence Test [1] [1] http://users.dsic.upv.es/proy/anynt/measuring.pdf http://users.dsic.upv.es/proy/anynt/measuring.pdf
zump 11y ago
Is anyone else irked by the questions about AGI? It's a bunch of fluff put forward by hacks. We're a long way off.
- AndrewKemendo 11y ago
  Considering it's explicitly called out in their charter [1]: It's hard to fathom how much human-level AI could benefit society, and it's equally hard to imagine how much it could damage society if built or used incorrectly. and basically the entire reason the project exists in the first place is to try and get a technical handle on AGI risks. I would have to say it makes perfect sense. We're only a long way off if we continue to put tiny dollar figures into it. You should be thinking about it more like the Nucelar issue. Would you rather more or less thought be put into it? [1] https://openai.com/blog/introducing-openai/ https://openai.com/blog/introducing-openai/
- chriswarbo 11y ago
  I think there are legitimate research questions and concerns about AGI; unfortunately there's also a lot of fluff surrounding the area (e.g. singularity stuff, doomsday scenarios, etc.). The way I see it, there's only one conceptual barrier to cross between current AI and "AGI-like" technology, which could be summed up as 'models which take themselves into account'. Whilst it's trivial to have software modify itself, we don't have good models for predicting what those modifications will do (in a way which is more computationally efficient than just running them). An analogy is how encoding programs as ANNs lets us perform gradient descent, which we couldn't do if we encoded them as e.g. string of Java code. If we find a powerful software model which allows efficient prediction with white-box self-references, then I think lots of progress will be made quite quickly.
meeper16 11y ago
How does one monetize their own individual breakthroughs if they develop something groundbreaking in AI and involved with OpenAI?

130 points by sushirain 11y ago

meeper16 11y ago

Any chance at asking questions here? Whoops, guess I just did...

gdb 11y ago
waves
- jk4930 11y ago
  Do you have info materials about OpenAI I could use in AI events? Will you work with 3rd parties to rise awareness and support, recruit talent and external collaborators (e.g. research groups)?
  - gdb 11y ago
    No particular materials right now besides the website. We expect to collaborate with many third parties over time. We're just getting started, so not sure on the specifics yet!
    - jk4930 11y ago
      Okay. May I write you an email to establish the connection? I intended to contact OpenAI later this month, but now that we're talking...
      - gdb 11y ago
        Feel free to get in touch: gdb@openai.com!
        jk4930 11y ago
        I'll send you a mail tomorrow. Thanks.
  - nl 11y ago
    May I suggest Kaggle? The AllenAI Institute is currently running a pretty interesting contest there at the moment, with pretty direct links to OpenAI's mission. Of course Kaggle often degenerates into a "who can drive xgboost[1] the best", but some of the feature engineering is quite novel. A decent Winograd schema challenge would be pretty interesting. [1] https://github.com/dmlc/xgboost#whats-new https://github.com/dmlc/xgboost#whats-new
- emcq 11y ago
  Do you believe there is a limit to what models we can train with layers of differentiable units?

TeMPOraL 11y ago

Scott Alexander expressed some important concerns about the project[0]. Edited: <del>So far they've managed to label them as "coming from the LessWrong background" and subsequently dismiss via appeal to a strawman Paperclip Maximizer. It doesn't give me much confidence in them.</del> <ins>Nevermind. I didn't realize this comment was not made by an OpenAI representative. Also, we could use a strikethrough formatting tag on HN. 'dang?</ins> I hope they eventually address those points though. [0] - http://slatestarcodex.com/2015/12/17/should-ai-be-open/ http://slatestarcodex.com/2015/12/17/should-ai-be-open/

gdb 11y ago
To be clear, the following comment is what we wrote on the subject: https://www.reddit.com/r/MachineLearning/comments/404r9m/ama_the_openai_research_team/cysejas https://www.reddit.com/r/MachineLearning/comments/404r9m/ama.... The subsequent replies are not affiliated with OpenAI.
- TeMPOraL 11y ago
  Thanks, I missed that. I didn't notice the OpenAI flair. I apologize for mistakenly assigning this comment to your group.
gjm11 11y ago
The person who referred to "LessWrong background" and talked about paperclip maximizers wasn't (at least, so it looks to me) mocking or dismissing. The last paragraph of that comment does say "the LessWrong folks tend to be overly dramatic in their concerns" but goes on immediately to add "But they do have a point that the problem of controlling something much more intelligent than yourself is hard [...] and, if truly super-human intelligence is practically possible, then it needs to be solved before we build it".

AndrewKemendo 11y ago

I missed the AMA, but glad to see it here and it looks like there are some team members monitoring this thread so I'll throw some questions out there. I did my MS in AGI at the National Intelligence University with Ben Goertzel as my outside research advisor. My thesis was to determine what were defense implications of an AGI and who was making the best progress toward actually building one. Since then the DoD has started to take an interest in AGI, and in fact today, during my one weekend a month drill at the Pentagon, I had a great long conversation with Maj. General Seng [1] who is heading up efforts around implementing ISR systems with more autonomous capabilities and exploring how an AGI would be utilized in defense. One of our big open questions is what's the "stack" for AGI, conceptually? I didn't come to any conclusion on this and had to make a lot of assumptions to close out the research. I would be curious to hear the OpenAI team's thoughts on it. Will you all be coming to AGI 16 this year in New York? [1] http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Article/108644/major-general-jocelyn-m-seng.aspx http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Arti...

eli_gottlieb 11y ago
>Since then the DoD has started to take an interest in AGI, and in fact today, during my one weekend a month drill at the Pentagon, I had a great long conversation with Maj. General Seng [1] who is heading up efforts around implementing ISR systems with more autonomous capabilities and exploring how an AGI would be utilized in defense. Isn't that basically the most obviously bad idea ever, so terribly stupid that there have been several movies chronicling how massively bad an idea it actually is to build "AGI" for military goals?
- AndrewKemendo 11y ago
  I think you have been watching too many movies.
  - TeMPOraL 11y ago
    Movies sometimes get this thing right though. Don't forget that Dr. Strangelove pretty much turned out to be a documentary!

chegra 11y ago

1. Do you believe solving AI will come from a big company where the employees solving it would have little ownership of the company? 2. Currently, there is no solid test for Artificial General Intelligence. How much of a priority is it to create one? ------------------------------------------------------- Answering my own questions: 1. No, the incentive is just not there. 2. Currently, this is my main goal related to Artificial General Intelligence. One should know how far or near they are from creating Artificial General Intelligence.

AndrewKemendo 11y ago
Currently, there is no solid test for Artificial General Intelligence. How much of a priority is it to create one? There is of course no commonly agreed upon one, but in my research but the best one I found was the Universal Anytime Intelligence Test [1] [1] http://users.dsic.upv.es/proy/anynt/measuring.pdf http://users.dsic.upv.es/proy/anynt/measuring.pdf

zump 11y ago

Is anyone else irked by the questions about AGI? It's a bunch of fluff put forward by hacks. We're a long way off.

AndrewKemendo 11y ago
Considering it's explicitly called out in their charter [1]: It's hard to fathom how much human-level AI could benefit society, and it's equally hard to imagine how much it could damage society if built or used incorrectly. and basically the entire reason the project exists in the first place is to try and get a technical handle on AGI risks. I would have to say it makes perfect sense. We're only a long way off if we continue to put tiny dollar figures into it. You should be thinking about it more like the Nucelar issue. Would you rather more or less thought be put into it? [1] https://openai.com/blog/introducing-openai/ https://openai.com/blog/introducing-openai/
chriswarbo 11y ago
I think there are legitimate research questions and concerns about AGI; unfortunately there's also a lot of fluff surrounding the area (e.g. singularity stuff, doomsday scenarios, etc.). The way I see it, there's only one conceptual barrier to cross between current AI and "AGI-like" technology, which could be summed up as 'models which take themselves into account'. Whilst it's trivial to have software modify itself, we don't have good models for predicting what those modifications will do (in a way which is more computationally efficient than just running them). An analogy is how encoding programs as ANNs lets us perform gradient descent, which we couldn't do if we encoded them as e.g. string of Java code. If we find a powerful software model which allows efficient prediction with white-box self-references, then I think lots of progress will be made quite quickly.

meeper16 11y ago

How does one monetize their own individual breakthroughs if they develop something groundbreaking in AI and involved with OpenAI?