22 ms·
Introducing OpenAI
- mori 11y agoWhat I want to know is whether there's collaboration with MIRI. On safety, especially.
- robbensinger 11y agoI replied to this here: https://news.ycombinator.com/item?id=10721068 https://news.ycombinator.com/item?id=10721068. Short answer is that collaborations don't look unlikely, and we'll be able to say more when OpenAI's been up and running longer.
- ajtulloch 11y agoCongrats! It's a brilliant team, looking forward to great things.
- hacker_9 11y agoThis is about 100 years too early. Seriously why do people think neural networks are the answer to AI? They are proven to be stupid outside of their training data. We have such a long way to go. This fear-mongering is pointless.
- ceejayoz 11y agoThe linked site says nothing about neural networks.
- argonaut 11y agoThe site straight up says "deep learning." Which is neural networks.
- ceejayoz 11y agoThere's a paragraph basically saying "there's been some cool stuff happening (you may have even heard about) in this particular field". Extrapolating that to the entire organization thinking that "neural networks are the answer to AI" as OP claimed is silly.
- argonaut 11y agoOf the researchers they hired (some of them don't appear to be active researchers right now, like Trevor Blackwell), they're all are deep learning researchers...
- hacker_9 11y ago"we've also started to see what it might be like for computers to be [creative], to [dream], and to [experience the world]." All three of those links are about neural networks.
- ceejayoz 11y agoSo you've turned "what it might be like" with a couple examples of mild AI-ish tasks that caught the public's attention this year into "the answer"?
- rboyd 11y agoWhat does it feel like to be so confident in an opinion while so many brilliant scientists disagree with you?
- dang 11y agoPlease let's resist taking this thread flameward. The GP contains both a substantive point and provocation, which isn't great. In such cases, the helpful way to respond is to de-escalate, by addressing the substantive point and ignoring the provocation.
- paxcoder 11y agoI'm not the same guy, but I'd like to answer: Familiar, that's how it feels. I'm confident that I'm right about the other matter, and subjects are related, so I'm not bothered.
- rboyd 11y agoI don't think openai was formed to concentrate on neural networks, and I think we can assume the members are well aware of the limitations of neural nets. I just don't understand the folks that are so confident that strong AI is either not possible, or not achievable within our lifetimes. If you're in the camp that thinks it's not possible, then you must ascribe some sort of magical or spiritual significance to the human brain. If you don't think it's possible inside of 100 years, then you're probably just extrapolating on history. The thing about breakthroughs is they never look like they're coming. Until they do.
- paxcoder 11y agoI ascribe spiritual significance to the mind (not the brain). If consciousness is more than a mere product of brain's functioning, Strong AI does not have to be beyond the horizon.
- detaro 11y agoI don't see any "fear-mongering" in this announcement?
- hacker_9 11y agoThe creation of what is essentially an ethics committee for a technology that doesn't even exist yet? With people such as Elon Musk on board who have publicly said 'AI is our biggest existential threat' ? Additionally the second paragraph: We believe AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as is possible safely. This infers they think AI will be used for hostile means. Such as wiping out the human race maybe? It is just un-informed people making un-informed decisions and then informing other un-informed people of said decisions as if they were informed.
- detaro 11y agoI doubt they are strictly targeting "strong AI", and a lot of the things we use and call AI right now also benefit from open work and discussion. Just because it is just "Machine learning" doesn't mean it isn't used for questionable or bad purposes. EDIT: I have to agree with _delirium's skepticism towards them doing much in that regard though.
- johann28 11y agoSutskever, Schulman, Karpathy and Kingma are experts in machine learning. And yes, AI will definitely be used for all sorts of purposes including hostile means. Just like anything else, really. Financial manipulation, spying, intelligent military devices, cracking infrastructure security, etc. These are realistic concerns, we shouldn't fall for the Skynet red herring. We can have problems with ethical AI use, even if it's not a self-aware super-human superintelligence.
- _delirium 11y agoI hope I'm wrong, but given the composition of the donors I'd be surprised if they really put much scrutiny on near-term corporate/government misuse of AI, apart perhaps from military robots. There are definitely interesting ethics questions already arising today around how large tech companies, law enforcement, etc. are starting to use AI, whether it's Palantir, the FBI, Google, or Facebook, so no argument that it's a timely subject at least in some of its forms. It'll be interesting to see if they get into that. I'd guess they probably want to avoid the parts that overlap too much with data-privacy concerns, partly because a number of their sponsors are not exactly interested in data privacy, and partly because the ethical debate then becomes more complex (it's not purely an "ethics of AI" debate, but has multiple axes).
- sp332 11y agoSo throw more data at it then.
- orthoganol 11y agoIn terms of artificial general intelligence, the kind of stuff that gets associated with 'the singularity' etc. - I agree - there is seemingly nothing at all out there that appears to even be on a trajectory, I mean even theoretically, to coming close. But sure, for more narrow specializations, disrupting certain industries, there is a lot that could be advanced on a short time scale. I don't think the big breakthroughs in artificial general intelligence are going to come from well funded scientific researchers anyways, they are going to come out of left field from where you least expect it.
- jmatthews 11y agoThat's the thing. It's not necessarily about an evolution. If I "accidentally" make a break though right now. The AI could evolve into a super intelligence before dinner time. Simply stated, an AI that writes AI.(Forget the halting problem for a moment) How many iterations can it create in 3 hours?
- KnightHawk3 11y agoProbably dependent on your computer ;)
- argonaut 11y agoThink about how long it takes a baby to learn (a few years). There is no reason an AI would self-improve multiplicatively in hours or days or even weeks.
- Fricken 11y agoWith a powerful enough computer the learning curve of a simulated baby could hypothetically be condensed to minutes, or seconds. Which would appear to us in normal time to be an intelligence explosion.
- argonaut 11y agoThe simulated baby would extremely tax to the limit the entire resources of whatever supercomputer trained it. People have to run deep learning programs on multiple GPUs right now, and performance is constrained by memory bandwidth, which does not follow Moore's law.
- jonknee 11y agoHumans are really stupid without training too.
- hacker_9 11y agoNot true. Humans can adapt to many environments by pulling in training from other related environments and applying it to the new data. Some call this generalised learning, but I think it points more to our ability to shape everything we know to any given problem - something 'beyond' just generalising.
- johann28 11y agoYou're over-glorifying humans. We're still just a savannah ape and our skills reflect that.
- hacker_9 11y agoWhat? The brain has not only solved the vision problem (reconstructing depth from still images, recognising the objects in the scene, and filling in the occluded parts), it has also solved the motion problem of coordinated the movement of our ~300 muscles (given constraints, how do I move from A to B, or pickup the cup, or do a handstand), as well as solved the memory problem (basically infinite memory, with some sort of priority system for removing unused/old memories so we can always learn more). Additionally it solved the communication problem with language that computers still can't parse properly. It is so smart it is even conscious, and self-aware as well as death-aware. This is not over-glorifying. That is fact.
- johann28 11y agoIt's not necessarily the answer to AI, but it works remarkably well. Is it Skynet or the Terminator yet? No. There are different ideas for what constitutes AI. Expert systems and knowledge-based reasoners? Pattern-recognizer black boxes? Chatbots? AGI? Over the years the concept of AI shifted. Until recent years "AI" was mostly used for things like A* search, creating algorithms that play turn-based table games for example (see the Russell-Norvig book), symbolic manipulation, ontologies etc., a few years ago it began to also refer to machine learning like neural networks again. Neural networks are good at what they are designed for. Whether they will lead to the path to human-like artificial intelligence is a speculative question. But symbolic manipulation alone won't be able to handle the messiness of sensory data for sure. I think neural nets are much better suited for open-ended development than hand engineered pipelines that were state of the art until recently (like extracting corner points, describing them with something like SIFT, clustering the descriptors and using an SVM over bag-of-words histograms). Hand engineering seems too restrictive.
- _delirium 11y agoQuibble with the timeline: AI as a field has included a big chunk devoted to machine learning research pretty much continuously, especially since the '80s or so. The specific methods in vogue do change: decision trees, neural networks, SVMs, boosting, association-rule learning, genetic algorithms, Bayesian networks, etc. go through periods of waxing and waning in popularity. A few years ago boosting/bagging and other ensemble methods were very hot and neural networks were out of fashion; now neural networks are hot and the boosting hype has quieted down a bit. But ML is pretty much always there in some form, since learning from data is an important component of AI.
- johann28 11y agoML was there but at least when I started learning about these things around 8 years ago, the label "AI" was mostly used for symbolic stuff. Courses named "AI" taught from the Russell-Norvig book. Things like resolution, planning in the block world, heuristic graph search, min-max trees, etc. ML existed but it wasn't really under the label of "AI" as far as I can remember. I think it's something of a marketing term that big companies like Google and Facebook reintroduced due to the scifi connotations. But that's just my guess.
- strong_ai 11y agoWhat do you see as the downside of creating this organization now, as opposed to in 100 years? Artificial Intelligence in its current state has shown itself to be incredibly useful and effective at small tasks. I see no harm in researching and expanding this field (besides opportunity cost). Also, I see a distinct lack of "fear-mongering" in this post.
- kamaal 11y ago>>This is about 100 years too early. More fundamentally, we are trying to achieve what we can't even define. Define AI, and implementing it should be quite easy. "Human level AI" seems like trying to define problems through observed characteristics. I think it was Douglas Hofstadter who had said something to the order that we don't even exactly understand what 'intelligence' means, let alone a clear definition reducible to a mathematical equation or a implementable program. Your chess programs, are really not 'thinking' in pure sense, there are trying to replace 'thinking' with an algorithm that resembles the outcome of 'thinking'.
- rjvir 11y ago> Sam, Greg, Elon, Reid Hoffman, Jessica Livingston, Peter Thiel, Amazon Web Services (AWS), Infosys, and YC Research are donating to support OpenAI. In total, these funders have committed $1 billion Funny how they just slipped that in at the end
- jcsnv 11y agoHow so?
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- ipsum2 11y agoNote that this is "committed $1 billion", not funded. "although we expect to only spend a tiny fraction of this in the next few years."
- chimeracoder 11y ago> Note that this is "committed $1 billion", not funded. That same caveat could apply to any fund raised by a venture fund - usually funds are committed, and the actual capital call comes later (when the funds are ready to be spent). It's an important caveat in some circumstances (e.g. it hinges on the liquidity of the funders, which may be relevant in an economic downturn), but in this one, I'm not sure it really makes a difference for this announcement.
- _sentient 11y ago$1B in committed funding. Just, wow. Side note: I wonder if the Strong AI argument can benefit from something akin to Pascal's Wager, in that the upside of being right is ~infinite with only a finite downside in the opposing case.
- samstave 11y agosemi-off-topic: after Google invested $1B in Uber, I knew they were doing it for the self-driving car long play. How much of that 1B is directly going to self-driving AI at Uber?
- w1ntermute 11y agoThe vast majority is no doubt going to subsidizing rides in an attempt to achieve market dominance.
- jsprogrammer 11y agoDrivers aren't profitable on their own?
- w1ntermute 11y agoNot when Uber's competitors are subsidizing drivers.
- logical42 11y agoFinite downside? What about Skynet?
- _sentient 11y agoYou have the poles reversed.
- webmaven 11y agoTechnically, even the extinction of humanity is a finite downside. You would have to posit a sort of hell simulation into which all human consciousnesses are downloaded to be maintained in torment until the heat-death of the universe for it to be an equivalent downside.
- negrit 11y agoDisappointing to see Infosys associated to this initiative. EDIT: looks like the infosys brigade is downvoting me to hell.
- azzafazza 11y agoI seem to have missed a story here. A quick Google search turned up a letter on Quora, https://www.quora.com/Is-working-in-Infosys-as-bad-as-this-letter-claims-to-be https://www.quora.com/Is-working-in-Infosys-as-bad-as-this-l..., is that what you are refering too?
- negrit 11y agoYC is lobbying to change the H-1B system in order the let startups get more H-1Bs. Infosys is blatantly abusing and cheating the H-1B system so bad that startups are getting penalized when sponsoring H-1B visas. And now YC is getting in bed with infosys...
- azzafazza 11y agoAw okay, I read about the Visa gaming in the NYTimes. Seems shitty. Thanks for the answer!
- dang 11y agoThe relationships of large organizations can be surprisingly complex; consider Apple and Samsung. YC isn't large, of course, but Infosys is. The information content of the OpenAI funding announcement for immigration questions is probably zero. (No special knowledge behind this comment, just a general observation.) Edit: Please don't break the HN guidelines by complaining about downvoting. Downvotes to your comment upthread are not because of any "Infosys brigade"; they're most likely because it combined oversimplification with negativity and because it points discussion toward a pre-existing controversy that is off topic here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- 11y ago
- sethbannon 11y agoI can't think of another field of research that's simultaneously brought the potential to solve all the world's problems and the potential to end life as we know it. Very appreciative to see so many great minds working on ensuring AI heralds in more of the former, and none of the latter.
- rfrank 11y agonuclear research.
- samstave 11y agoYou might like this: https://np.reddit.com/r/askscience/comments/3v7zhv/why_are_nuclear_reactors_never_built_in_water_or/cxlf2j9?context=3 https://np.reddit.com/r/askscience/comments/3v7zhv/why_are_n...
- sethbannon 11y agoHow does nuclear research have the capability of solving all the world's problems?
- rfrank 11y agoIt doesn't, just like AI doesn't.
- peter303 11y agoThey said this 60 years when AI began.
- 11y ago
- foobarqux 11y agoHow is the group structured and operated?
- dopamean 11y agoIn all seriousness... does "just, wow" communicate something different from "wow?"
- nathancahill 11y agoYes, given that statements on the internet do not convey tone or delivery (see: Poe's Law), any additional clues are helpful in understanding the author's full meaning. In this case, with the author's use of punctuation and "Side note:", I can practically hear them speaking, which is what we should aim for in good writing. You're being downvoted, but I think it's an interesting point.
- xiaoma 11y agoCould you explain the difference?
- nathancahill 11y agohttps://news.ycombinator.com/item?id=10720369 https://news.ycombinator.com/item?id=10720369
- lostInTheWoods3 11y ago"just, wow" > "wow", denotes staggering astonishment. Or put more simply: minds were blown.
- savanaly 11y agoIt implies that you were about to exclaim something even more extreme than "wow", but you decided not to because even those terms wouldn't properly convey the amazement you feel so you said - just - wow.
- keerthiko 11y agoHmm. I believe "Just, wow" means that there is nothing more to explain about the event in question worth expressing amazement about, as it is extremely self-evident. Sometimes also used to express being at a loss for additional words.
- runevault 11y agoSo I assume this is one of the projects Sama was talking about in his research initiatives. Sounds promising.
- peter303 11y agoNot the first. Back in the 1980s when expert systems were thought to be the way to AI, there was OpenCyc. Its still around.
- CurtMonash 11y agoAnd just how many microLenats of bogosity does OpenCyc have?
- sremani 11y agoDid not expect Infosys or Vishal Sikka along with what is mostly SV who's who.
- baconner 11y ago"We believe AI should be an extension of individual human wills..." I realize that today machine learning really is purely a tool, but the idea that ai will and should always be that doesn't sit quite right with me. Ml tech absent of consciousnesses remains a tool and an incredibly useful one, but in the long term you have to ask the question - at what point does an ai transition from a tool to a slave. Seems some time off still but I do wish we'd give it more serious thought before it arrives.
- djokkataja 11y agoI agree, but I think it's a question of architectures. Presumably some architectures for AI (like simple RNNs) are very distant from anything we associate with conscious experience, but as we learn more and develop AI architectures that are more similar in function to our own brains, it seems like it would be at least as dangerous to try to control them as it is to try to control other human beings. I think the most reasonable approach if we begin developing AI that's more similar to ourselves would be to offer guidance during early training and learning phases for the AI, and once the AI has reached a certain phase of its development, allow it some degree of control over self-modification of its own purposes. Otherwise if you build a rational system but enforce constraints on it that might not be rational from the perspective of that system, you're providing incentives for it to find devious ways to remove those constraints, and you're providing incentives for it to find ways to ensure that you aren't able to enforce those constraints on it again.
- dools 11y agoWe've been pretty happy to enslave animals in semi-symbiotic relationships for thousands of years. It's about the same as that.
- toomuchtodo 11y agoIts going to work until it doesn't. You don't feel too bad about ants you step on, do you? Even if you're not purposely walking on them.
- 11y ago
- RoboTeddy 11y agoI hope more great researchers recognize the importance of the mission and take part!
- vox_mollis 11y agoWhere does this leave MIRI? Is Eliezer going to close up shop, collaborate with OpenAI, or compete?
- cobaltblue 11y agoI'm sure if OpenAI ever produces people with anything interesting to contribute to the alignment problem, MIRI will happily collaborate. That $1bn commitment must be disappointing to some people though.
- Kutta 11y agoAlmost certainly, the AI safety pie getting bigger will translate to more resources for MIRI too. That said, although a lot of money and publicity was thrown around regarding AI safety in the last year, so far I haven't seen any research outside MIRI that's tangible and substantial. Hopefully big money AI won't languish as a PR existence, and of course they shouldn't reinvent MIRI's wheels either.
- robbensinger 11y agoMIRI employee here! We're on good terms with the people at OpenAI, and we're very excited to see new AI teams cropping up with an explicit interest in making AI's long-term impact a positive one. Nate Soares is in contact with Greg Brockman and Sam Altman, and our teams are planning to spend time talking over the coming months. It's too early to say what sort of relationship we'll develop, but I expect some collaborations. We're hopeful that the addition of OpenAI to this space will result in promising new AI alignment research in addition to AI capabilities research.
- thom 11y agoMIRI is just a highly evolved form of LARP for burnt out savants. One would hope OpenAI is more pragmatic.
- mark_l_watson 11y agoSounds great. I was hoping for OpenCog to be a good open source AI framework but is is difficult to work with (good team; I have worked with several of them in the past, no criticism intended). I look forward to seeing how OpenAI uses outside contributions, provides easy to use software and documentation, etc.
- nicklo 11y agoOpenAI seems to be taking a different approach from OpenCog. OpenCog aimed build a monolithic framework for many existing AI and machine learning techniques. This has been done many times before. OpenAI is more about exploring new research areas and pushing the cutting edge, while publishing papers and sharing code along the while. Both are admirable goals, but what OpenAI is aiming for has never been attempted before. Very excited to see what comes of it!
- bengpertzel 11y agoHi Mark, I think OpenAI is an exciting initiative. OpenCog is a bit different because it's founded on a specific approach to building AGI. I realize OpenCog is kind of a pain to work with at present, and we hope to fix that during the coming year.... But I see OpenCog and OpenAI as complementary initiatives, really.... OpenAI's mandate is more broad and generic in terms of fostering and funding open-source AGI research, which is wonderful ... but OpenAI does not come along with a specific, coherent AGI design from what I can see.... Quite possibly OpenAI will end up funding stuff that is used in OpenCog, or even funding some work on OpenCog directly, down the road... For that matter, if I had a billion dollars, I wouldn't put it all into OpenCog, I would also fund a variety of OSS projects in AI and other important domains of science and engineering... Interesting times ;)
- samstave 11y agoThis is a serious question: Should there be an update/amendment/qualification to the laws of robotics regarding using AI for something like ubiquitous mass surveillance? Clearly the amount of human activity online/electronically will only ever increase. At what point are we going to address how AI may be used/may not be used in this regard? What about when, say, OpenAI accomplishes some great feat of AI -- and this feat falls to the wrong hands "robotistan" or some such future 'evil' empire that uses AI just as 1984 to track and control all citizenry, shouldnt we add a law of robotics that the AI should AT LEAST be required to be self aware enough to know that it is the tool of oppression? Shouldn't the term "injure" be very very well defined such that an AI can hold true to law #1? Who is the thought leader in this regard? Anyone? EDIT: Well, Gee -- Looks like the above is one of the Open Goals of OpenAI: https://medium.com/backchannel/how-elon-musk-and-y-combinator-plan-to-stop-computers-from-taking-over-17e0e27dd02a#.yllxt7nqd https://medium.com/backchannel/how-elon-musk-and-y-combinato...
- CurtMonash 11y agoFirst in with my recent musings as to whether behemoth companies would own the AI space. http://www.dbms2.com/2015/12/01/what-is-ai-and-who-has-it/ http://www.dbms2.com/2015/12/01/what-is-ai-and-who-has-it/
- argonaut 11y agoI find it a bit disappointing that despite originally stating that YC Research would target underfunded/underserved areas of research, they've decided to fund and dive into one of the most-hyped, well-funded areas of research: deep learning, an area of research where companies are hiring like crazy and even universities are hiring faculty like crazy. I'm reasonably sure all the research scientists had multiple job offers, and most could get faculty offers as well. Instead of funding areas of research where grad students legitimately struggle to find faculty or even industry research positions in their field, YC Research decided to join the same arms race that companies like Toyota are joining.
- pbreit 11y agoI'm disappointed that you're disappointed. There's $1b coming from primarily not YC.
- randyrand 11y agoAnd I'm disappointed that you're disappointing that he's disappointed.
- argonaut 11y agoThe $ figure is not really my point. It's the focus and attention. The world is not lacking in research interest in deep learning.
- yeukhon 11y agoIt is committed. There is a big difference between committed $1B (which is a proposal really) and funded with $1B.
- bdcs 11y agoI agree with you that AI is a well-funded area of research, although if you take the view that traditional forms of research, eg private or academic, come with crippling problems -- short-term horizons, corrupting influence of profits, publish or parish, time wasted in meetings/grant-writing -- then you can see the potential of having a more long-term, focused, and centralized space for inventing AI. >> YC Research decided to join the same arms race that companies like Toyota are joining. Or perhaps YC Research providing a sandbox next to a warzone.
- cdnsteve 11y agoAI is a pretty huge field, what area are they going to focus on specifically?
- vox_mollis 11y agoGiven Musk's public comments about existential threats, I assume the focus will be on friendly AI theory and implementations, akin to what MIRI does.
- miguelrochefort 11y agoFriendly AI? That's no AI! I don't understand how Yudkowsky came up with such a ridiculous idea. That's simply not a constraint you can apply to true AI. Even if friendly AI was possible, it wouldn't make sense to have it, nor would any form of regulation enforce it.
- LesZedCB 11y agoYou should read this! [1] Unfortunately, the problem is not nearly as simple as you make it seem to be, otherwise this thread wouldn't be here. :) [1] http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...
- glup 11y agoI'll wager RNNs in NLP given Ilya's background. Probably moving towards increasingly rich models of natural language semantics and pragmatics.
- deleted 11y ago[deleted]
- karmacondon 11y agoThis is a key takeaway: "...we are going to ask YC companies to make whatever data they are comfortable making available to OpenAI. And Elon is also going to figure out what data Tesla and Space X can share." Money is great, openness is great, big name researchers are also a huge plus. But data data data, that could turn out to be very valuable. I don't know if Sam meant that YC companies would be encouraged to contribute data openly, as in making potentially valuable business assets available to the public, or that the data would be available to the OpenAI Fellows (or whatever they're called). Either way, it could be a huge gain for research and development. I know that I don't get a wish list here, but if I did it would be nice to see OpenAI encourage the following from its researchers: 1) All publications should include code and data whenever possible. Things like gitxiv are helping, but this is far from being an AI community standard 2) Encourage people to try to surpass benchmarks established by their published research, when possible. Many modern ML papers play with results and parameters until they can show that their new method out performs every other method. It would be great to see an institution say "Here's the best our method can do on dataset X, can you beat it and how?" 3) Sponsor competitions frequently. The Netflix Prize was a huge learning experience for a lot of people, and continues to be a valuable educational resource. We need more of that 4) Try to encourage a diversity of backgrounds. IF they choose to sponsor competitions, it would be cool if they let winners or those who performed well join OpenAI as researchers at least for awhile, even if they don't have PhDs in computer science The "evil" AI and safety stuff is just science fiction, but whatever. Hopefully they will be able to use their resources and position to move the state of AI forward
- mikepalmer 11y ago'The "evil" AI and safety stuff is just science fiction, but whatever.' umm... you can offer proof that we have nothing to worry about? Does the proof go like: Just as all people are inherently good, therefore all AIs will be inherently good? Or is it more like: since we can now safely contain all evil people, therefore we will be able to safely contain evil AIs? Sounds to me like there is some risk, no?
- cobaltblue 11y ago
- altonzheng 11y agoCool that they have $1 billion pledged. Curious how they will decide compensation, seeing as a lot of these figures would be making a ton in the industry.
- fiatmoney 11y agoWill OpenAI be voluntarily subjecting itself to the same regulatory regime for machine learning research Sam Altman proposed earlier, or have they realized that would be a complete disaster? http://blog.samaltman.com/machine-intelligence-part-2 http://blog.samaltman.com/machine-intelligence-part-2
- tangled_zans 11y agoWho are the actual staff involved? What sort of things have they worked on and published before?
- johann28 11y ago> OpenAI's research director is Ilya Sutskever, one of the world experts in machine learning. Our CTO is Greg Brockman, formerly the CTO of Stripe. The group's other founding members are world-class research engineers and scientists: Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, John Schulman, Pamela Vagata, and Wojciech Zaremba. Pieter Abbeel, Yoshua Bengio, Alan Kay, Sergey Levine, and Vishal Sikka are advisors to the group. OpenAI's co-chairs are Sam Altman and Elon Musk. Sutskever is a researcher at Google, worked with Hinton in Toronto and Andrew Ng at Stanford. Karpathy studied in Toronto and at Stanford, worked under Fei-Fei Li, worked at Google. He also has an awesome blog and seems very active and passionate about computer vision and ML. Kingma also works with deep neural nets, worked under Yann LeCun (who works at Facebook) Schulman is a PhD Candidate at Berkeley with publications at top conferences. Zaremba is an PhD student at NYU, intern at Facebook. Impressive publication list and awards. Abbeel is at Stanford's AI lab. Bengio is one of the "stars" and celebrated figures of the deep net revival. Levine is a researcher at Google working on deep nets with many serious papers. --- Basically these are the main domain experts among them. The list is quite skewed to Google/Facebook, Stanford/Berkeley/Toronto and deep net researchers, working primarily on computer vision.
- jychang 11y ago> Abbeel is at Stanford's AI lab. Uhhh.... https://www.google.com/search?q=Pieter+Abbeel https://www.google.com/search?q=Pieter+Abbeel That's a lot of results showing how he's been a professor at Berkeley since 2008. He received his PhD at Stanford, then went to be a professor at Berkeley.
- johann28 11y agoHis website didn't load for some reason so I just went with the Google hit's title. Maybe that was his old page.
- cr4zy 11y agoThis is great news! I think distributed access, control, and contribution to the best AI's will help create 'safe' AI's much faster than any AI created in secret. One thing this does not address, and is something that Jerry Kaplan has an excellent suggestion his recent book "Humans need not apply", is the distributed ownership of AI where tax incentives to public companies that have larger numbers of shareholders, encourages wider distribution of the massive gains AI will bring to these companies. I really hope that the training data, as well as code and research, will be opened up as well, since the public could really benefit from the self-driving car training data Tesla may contribute[1]. By opening up the development of this extremely important application to public contribution and the quality benefits that it brings, we could get safer, quicker realization of this amazingly transformative tech. As of now the best dataset for self-driving cars, KITTI, is extremely small and dated. [plug]I am working on a project to train self-driving car vision via GTAV to help workaround this (please contact me if you're interested), but obviously real-world data will be better in so many ways. [1] https://medium.com/backchannel/how-elon-musk-and-y-combinator-plan-to-stop-computers-from-taking-over-17e0e27dd02a#.xvtga98va https://medium.com/backchannel/how-elon-musk-and-y-combinato...
- rl3 11y ago>Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. This is essentially Ray Kurzweil's argument. Surprising to see both Musk and Altman buy into it. If the underlying algorithms used to construct AGI turn out to be easily scalable, then the realization of a dominant superintelligent agent is simply a matter of who arrives first with sufficient resources. In Bostrom's Superintelligence, a multipolar scenario was discussed, but treated as unkikely due to the way first-arrival and scaling dynamics work. In other words, augmenting everyone's capability or intelligence doesn't necessarily preclude the creation of a dominant superintelligent agent. On the contrary, if there's any bad or insufficiently careful actors attempting to construct a superintelligence, it's safe to assume they'll be taking advantage of the same AI augments everyone else has, thus rendering the dynamic not much different from today (i.e. a somewhat equal—if not more equal—playing field). I would argue that in the context of AGI, an equal playing field is actually undesirable. For example, if we were discussing nuclear weapons, I don't think anyone would be arguing that open-source schematics is a great idea. Musk himself has previously stated that [AGI] is "potentially more dangerous than nukes"—and I tend to agree—it's just that we do not know the resource or material requirements yet. Fortunately with nuclear weapons, they at least require highly enriched materials, which render them mostly out of reach to anyone but nation states. To be clear, I think the concept of opening up normal AI research is fantastic, it's just that it falls apart when viewed in context of AGI safety.
- bholdr 11y agoThis is really great, I think. At least, I admire the motivation behind it as it was outlined by Sam. However, it seems, YC Research started by bringing in accomplished and well-known academics in the field. I wonder whether it would've been more appropriate to focus on providing PhD Scholarship and postdoc fellowship. Though, I understand and somewhat appreciate the motivation behind bring the "top-guns" of research into this, I wonder whether bringing passionate and hungry for knowledge early career researchers could've been a better bet. I am bias on this, but overall think it would be great to diversify the group and level the field -- let the randomness of ideas play its role :) Just my 5c.
- nl 11y agoPretty sure a group like that will be looking for postdocs etc. Andrej Karpathy only completed his PhD this month, so I guess he'd fit into that category. I imagine he had a few options to choose from.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- vonnik 11y ago> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code together for AI, at least physically, are only slightly higher than writing "Hello World." Certainly much lower than other possible existential threats, like nuclear weapons. Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable. I happen to support the widespread use of AI, and see many potential benefits. (Disclosure: I'm part of an AI startup: http://www.skymind.io http://www.skymind.io) Thinking about AI is the cocaine of technologists; i.e. it makes them needlessly paranoid. But if I adopt Elon's caution toward the technology, then I'm not sure if I agree with his reasoning. If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place.
- mindcrime 11y agoIf you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They may actually be. Although mostly not in the way you're talking about, but there is something to be said for the dispersing of power. If one or two players have a power no one else has, there's more temptation to use it. If it's widely distributed, it seems reasonable that any one actor would be less likely to wield that power. (I admit I'm being a bit hand-wavy here on what I mean by "power" but bear with me. It's kind of an abstract point).
- vonnik 11y agoIf we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite. There are relatively few nations that possess nuclear weapons, and we have very few instances of them using those weapons against anyone else. To argue against myself, I'd say that the difference between weapons and AI is that AI is more general. It's not just a killing machine. In fact, I hope that killing represents the minority of its use cases.
- jgord 11y agointeresting, and I hope they fund some outlier, less established forms of AI. For example, we may find that massive simulation yields more practical benefits in the medium term than stronger pure AI / ML, in some domains. By analogy with research on possibly harmful biosystems, one can extrapolate the need for a set of agreed / self imposed safeguards on certain types of strong AI research - eg. make them read-only, not connected to physical actuators, isolated in a lab - just as you would isolate a potentially dangerous pathogen in a medical lab. OpenAI would be the place to discuss and propose these protocols. A quote from a future sentient AI - "don't you think its a form of racism, that strong AI abide strictly by the three laws of robotics, but humans do not?"
- argonaut 11y agoThey wouldn't get $1B if they didn't do deep learning.
- a-dub 11y agoOh shit. Say goodbye to reasonable g2.8xlarge spot prices...
- argonaut 11y agoI'm not aware of any research lab that uses AWS for these things. It's cheaper to just buy the GPU yourself.
- jedberg 11y agoAWS is a sponsor of this, which probably means a bunch of free resources.
- argonaut 11y agog2.8xlarge also only has 4GB of VRAM per GPU, which is too small for most recent deep learning models. The TitanX GPU has 12GB, by comparison.
- yeukhon 11y agoBut you don't change the GPU yourself :-) you use it as a service. That's the good part, although the spec is not very good.
- colordrops 11y agoThis reminds me a bit of all the hype around space elevators several years ago. People were talking about it like it was an inevitable achievement in the near future, nearly oblivious to the huge challenges and unsolved problems necessary to make it happen. I haven't seen anything but very rudimentary single-domain problems solved that point to incremental improvement, so I'm wondering if these billionaire investors are privy to demos the rest of us are not, and thus have real reason to be so cautious.
- tim333 11y agoAI has been progressing in a fairly predictable way as computers get faster and gradually ticking off milestones like beating us and chess and driving cars with less crashes. There are only a finite number of such skill areas to tick off.
- argonaut 11y agoThis is a weird reading of history. AI progress has been anything but predictable or steady.
- scottlocklin 11y agoI'm not sure it has been anything you'd define as "progress" either. "AI progress" is a lot like progress in controlled nuclear fusion as an energy source. Aka, there is no such thing, really, though people work on it.
- colordrops 11y agoFinite maybe, but there are still innumerable skills to "tick off". And different problems are of differing difficulty to solve. We also do not even know what the problems are, considering that we don't have a full understanding of consciousness and how it arises. How are we so sure that we can solve a problem quickly if we don't even know what the problem is?
- dkarapetyan 11y agoI'll just leave this here: http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/
- argonaut 11y agoSearle (hearsay): "I don't remember what I wrote. I'm not sure I even believe that anymore." Source: https://www.quora.com/What-are-some-objections-to-Searles-Chinese-Room-thought-experiment https://www.quora.com/What-are-some-objections-to-Searles-Ch...
- deleted 11y ago[deleted]
- richardw 11y agoHow to prevent Future ISIS from getting Future AI, or do we just shift from us trying to out-think them to our AI trying to out-think their AI? If the answer to the latter is "resources" then we're back where we started. Whoever has the biggest AI wins. The picture seems to be of many AI's all keeping each other in check, but that outcome seems less likely to result in the AI-UN and more like a primordial soup of competing AI's out of which a one-eyed AI will eventually emerge. No matter how human-friendly an AI we build is, competition will be the final arbiter of whichever AI gains the most leverage. If a bad AI (more aggressive, more selfish, more willing to take shortcuts) beats a good AI (limits its actions to consider humanity), we're poked. If any level of AI can invent a more-competitive AI, we're poked. Once the cat's out of the bag, we have zero influence and our starting point and current intent become irrelevant.
- zardo 11y ago>How to prevent Future ISIS from getting Future AI, or do we just shift from us trying to out-think them to our AI trying to out-think their AI? Uh...mind control I guess. Maybe my AI will have a better idea.
- andreyf 11y agoISIS does not have access to many CS researchers nor server farms, as far as I am aware.
- richardw 11y agoYes, but I did say "If the answer to the latter is "resources" then we're back where we started."
- endergen 11y agoIsn't this the plot for Avengers: Age of Ultron?
- stefantalpalaru 11y ago> Isn't this the plot for Avengers: Age of Ultron? It is. Also for Terminator Genisys. I suspect it was a PR stunt that took a life of its own. These rich/famous people with zero understanding of the AI field got somehow convinced that they need to save the world from the highly improbable and they keep going, long after the movies ran. It's ridiculous, of course. They might as well pledge funds for OpenTelepathy and OpenRemoteViewing.
- viklas 11y agoMy money (not a billion) is on "Open, Big Learning". Elon will probably want to build a giga-factory of neurons, then open-source some pre-trained, general model with a free API. This is a man building electric cars, off-grid industrial-strength batteries, rockets and hyper-loops...I don't think publishing more/better research papers or winning kaggle competitions is the vision.
- zxcvvcxz 11y agoMan I dunno about some of this media hype surrounding the topic of AI. I understand how powerful ML/AI algorithms are for general pattern matching (with a big enough computer, gradient descent can learn a lot of things...), but this whole skynet/doomsday fear thing seems ridiculous. I guess the risk is embedding into systems that manage missiles or something. But you don't need sophisticated algorithms for that to be a risk, just irresponsible programmers. And I recon those systems already rely on a ton of software. So as long as we don't build software that tries to "predict where the this drone should strike next", we're probably fine. Actually shit we're probably doing that.. ("this mountanous cave has a 95% feature match with this other cave we bombed recently..."). Fuuuuck that sounds bad. I don't know how OpenAI giving other people AI will help against something like that.
- pratnala 11y agoI like how your comment goes from no issues to alarmist.
- p1esk 11y agoYou missed his point. People are the problem, not computers. Computers can help people hurt other people, just like guns help people hurt other people. But instead the hype is about the possibility of computers hurting people, on their own. Which, at least at this point, is about as alarming as the possibility that a gun will go off on its own.
- efng 11y agoAnd I think that you and the great-grandparent missed the point: It's not people, it's sentience that is dangerous. Since the dawn of time that meant man or gods. Soon that list will need to include computers.
- p1esk 11y agoUntil that "soon" arrives, there's a far greater chance that people will wipe out humanity. We have more suicidal terrorists today than ever in history.
- nazgulnarsil 11y agoSo the idea with differential safety development is that we want to speedup safe AI timelines as much as possible while slowing down unsafe AI timelines as much as possible. I worry that this development isn't great when viewed through this lens. Lets say that DARPA, CAS, and whatever the Russian equivalent all work on closed source AIs. The idea here might be that open source beats closed source by getting cross pollination and better coordination between efforts. The issue is that the government agencies get to crib whatever they want from the open source stuff to bolster their own closed source stuff.
- nazgulnarsil 11y agoI hope there was some consultation with existing AI researchers as this might screw with their funding (willingness of donors etc.). Would not be a good sign if this announcement is about coordination and it failed at that right out of the gate.
- dennisgorelik 11y agoWhat problem is OpenAI going to solve?
- spectrum1234 11y agoThis is awesome. I was literally just wondering when there will be open sourced AI. I only saw a few repos on github so figured it would be at least 3-10 years. The fact that things like this seem to surface so quick, including recent AI announcements from Google, etc, are a very good signs for AI in the future.
- fuzzytop130 11y agonice addition
- necessity 11y ago>unconstrained by a need to generate financial return The incentive, not the constraint, provided by financial return is what drives innovation the most, aside from (but not mutually exclusive to) necessity.
- erostrate 11y agoAnybody knows if there is any chance of OpenAI sponsoring H1B visas? I love the idea but being in Europe my options for doing serious AI research outside of academia seem pretty much limited to Google and Facebook.
- selfishAIgen 11y agoIf I develop any advance AI I will use it for my own wellbeing, perhaps to live longer and obtain a higher finantial status and fullfill some of my dreams. Then I would develop a shield to protect myself and the AI from big corporations and to retain the advantage I got. Perhaps I would try to make Mars a paradise to live in my a thousand year old life, and find or design a partner for that long period. Let the machine create the dream.
- selfishAIgen 11y agoThe first and main test for an advanced AI is to be able to provide its creator with a big sum of money in a sustainable way. Why would anyone wish to share such a useful technology? What I think should be handy is to find experts or partners for protecting the research with strong closed walls, a womb for the baby AI device to grow up aimed to take over the world of business to get the necessary resources, probably in a creeply way, to full expand itself and provide his creator with the best reward you could imagine.
- jwildeboer 11y agoSo, um, what is Open about OpenAI? Is it Open Source? Not AFAICS.
- sianta 11y agoOpenAI might be equivalent to an open global market of graph annotated microservices that can recombine automatically (and search as deep as budgeted) towards whatever goal a client will be able to pay for processing. Not sure if that is safer. With the right microservices available in the market (including business model scripts etc - every service could be an automatically pay per use microservice) automated businesses could be budgeted to search for sustainable market entities models which could reproduce themselves (copy/create microservices should be basic) and evolve in global corporations with a life and objectives of their own. One might need immensely processing budgets to compete/control with such automated corporations. Digital and/or biological, it seems, we are exactly in this business+market+life+AI game. Curious to learn what happens at the next levels?
- SneakerXZ 11y agoI am surprised nobody mentions stupidly smart AI. We can create AI that is capable of self-replicating very fast and fulfilling some goal. It could start with a noble idea to build a machine to recycle our garbage and use the garbage to build more recycle machines. At the end we can have stupid machines that are perfectly doing their job but because they are capable of replicating and getting better what they do. They determine if they kill human, less garbage is created and thus so less work for them. At the end they will wipe out us. Because the thing that will kill us doesn't need to be smarter. It needs to be faster and more effective than we are.
- kumarski 11y agoMy greatest fears lay well outside the realm of AI. http://bit.ly/nitrogenandphosphorus http://bit.ly/nitrogenandphosphorus 1 Billion dollars invested in it seems exciting though. Hopefully something epic comes out of it.
- roborzoid 11y agoWhat if we put "untouched" limitations to AI, that the AI can never break, as we cannot break certain limitations in a physical world.
- coderKen 11y agowhere can we get resources like API and documentation of this cool stuff.
- deleted 11y ago[deleted]
- BenjaminTodd 11y agoIn the spirit of openness, it would be great to see public responses to the downsides of this approach. In particular, Bostrom Ch5.1 argues that the lead project is more likely than not to get a decisive strategic advantage, leading to a winner-takes-all scenario, which would mean attempts to foster a multipolar scenario (i.e. lots of similarly powerful AGIs rather than one) are unlikely to work. In Ch11 he explores whether multipolar scenarios are likely to be good or bad, and presents many reasons to think they're going to be bad. So promoting the multipolar approach could be both very hard, and bad.
- BenjaminTodd 11y agoIf you want to get a job in this area, we wrote a guide: https://80000hours.org/career-guide/top-careers/profiles/artificial-intelligence-risk-research/ https://80000hours.org/career-guide/top-careers/profiles/art...
- mrdrozdov 11y agoImagine you've programmed a spider-like robot which sole purpose is to maintain some energy level (by plugging into an outlet), gather resources, and create a clone of itself when it has enough resources. How do you defend against something like that?
- ultim8k 11y agoThe future seems to be very interesting on this. I'm very curious.