16 ms·
Open Philanthropy Project awards a grant of $30M to OpenAI
- vpontis 10y agoThat's awesome! Open Philanthropy reminds me of https://80000hours.org/ https://80000hours.org/. In their relationship disclosure: > OpenAI researchers Dario Amodei and Paul Christiano are both technical advisors to Open Philanthropy and live in the same house as Holden. In addition, Holden is engaged to Dario’s sister Daniela. This is so tangled. I don't mean it as a criticism as I'm sure a lot of SV investments would have a much longer Relationship Disclosure sections. So props to them for including this.
- rspeer 10y agoWell, I would use it as a criticism. This is a tangled web of like-minded people giving money to each other and calling it charity. Some people conflate this with Effective Altruism, which I think sucks. Compared to the rigorous work done by GiveWell, there's no way to tell if this is effective, or even altruistic. It's just people assuming that the world will be a better place if more people who think like them have money, an assumption held by basically everyone everywhere.
- engx 10y agoIt seems objectively better for some people to have access to money -- in this case the GiveWell founder. He's done great work there and now he's helping OpenAI all via a Facebook founder's billions.
- rspeer 10y ago"It seems objectively better for some people to have access to money" is classism in one sentence. And I won't give people a pass because of the thing they co-founded and then forked. I can appreciate Wikipedia and believe that Larry Sanger's fork of it was dumb. I can appreciate GiveWell and think that Open Philantropy is corrupt.
- engx 10y agoAnd your suggestion about classism is moral relativism. Who is corrupt? The guy giving away $8 billion or the one who founded GiveWell? And one of their focuses is criminal justice reform and trying to improve prison conditions. If you want to look at something truly corrupt, look at the criminal justice system in the U.S.
- jaibot 10y agoHolden's dedicated a huge chunk of his career to moving hundreds of millions of dollars to alleviating poverty and disease. He's one of the founders of the effective altruism movement. You may disagree with his decision here, but to dismiss his efforts, saying it "sucks" and isn't "effective, or even altruistic", while ignoring the extensive public writing he's done on the subject that led him to these views and strawmanning his position as nepotism, is just awful. "I disagree with the arguments presented, for these reasons" - cool. If you think the grant isn't a good idea, make an argument for that. "This person who has dedicated their life to doing as much good as possible is close to other people who also want to do as much good as possible, and their work has led to convergent viewpoints, therefore this isn't altruism" is cheap character assassination.
- hawkice 10y ago> strawmanning his position as nepotism So, you obviously feel strongly about this, but let me explain why your comments are less persuasive for those of us outside this subculture: The non-profit they donated to is (by any reading of their mission statement) an organization designed to create new technology that "will be the most significant technology ever created by humans" according to their own statements. It doesn't disburse cash or benefits to _anyone_, and actually pledges to keep some of the research secret, and "we expect to create formal processes for keeping technologies private when there are safety concerns" -- a situation the organization claims will happen, presumably regularly! Creating influential technology is typically done for-profit, and research is typically funded in ways much less open to individual favoritism (review boards are a great anti-corruption tool), and the results of that research are typically available to (among others) the people that fund it. There is a lot about this situation that a reasonable person would describe as unusual. In addition, all of these changes -- introducing more direct funding with less oversight, lack of access to results, lack of expectation of benefit to the targets of the charity -- all lend themselves to obscuring a fraud. That doesn't mean a fraud is present, but I'd be extremely aggressive about oversight. What kind of oversight are we getting? Well, right now they list one of their major goals as the "tricky" goal of figuring out of they're making any progress at all. I would not give this organization money. Dismissing these critiques as "character assassination" ignores the fact that I've only described aspects of the organization, not of the people involved, whom I have little information about.
- jackfoxy 10y agoOne of my favorite lines in Mad Men was Bert Cooper telling Don Draper Philanthropy is the gateway to power.
- deleted 10y ago[deleted]
- mankash666 10y agoI think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, MI anyone?
- acgourley 10y agoDo you disagree with any research looking to prevent existential risk to our species, or do you believe AI cannot become an existential risk?
- mankash666 10y agoI think existential risks to our species of much higher magnitude exist today. Climate change and pollution for instance. I think a "possible existential threat in the future" is a weaker case for philanthropy than existing ones.
- curiousgal 10y agoBoth of those issue cannot be solved by throwing money at them, especially the first. It requires political forces. Then again, the money could be used to buy politicians' votes in favor of environment-friendly policies. It's so blatant in the U.S. these days.
- arikr 10y agoIt's about expected value. A 0.0001% chance of reducing an extreme risk will have a very high expected value.
- rspeer 10y agoYou can justify lots of things with Pascal's wager, but most of them are religions.
- davmre 10y agoThere are always going to be more important causes, under any particular person's view of "more important", which depends very strongly on both subjective values and (in the case of AI alignment work) on precise probabilities of far-future outcomes. A dollar given to the Against Malaria Foundation will do a lot more good, in QALY terms, than the same dollar spent in Flint. And both dollars will do more (direct) good than a dollar given in funding to, say algebraic geometry research. Yet somehow we think it's important to fund all these things, and articles announcing new NSF grants for math research are not typically met with this kind of whatabout-ism.
- deleted 10y ago[deleted]
- seagreen 10y ago" When OpenAI launched, it characterized the nature of the risks - and the most appropriate strategies for reducing them - in a way that we disagreed with. In particular, it emphasized the importance of distributing AI broadly; our current view is that this may turn out to be a promising strategy for reducing potential risks, but that the opposite may also turn out to be true (for example, if it ends up being important for institutions to keep some major breakthroughs secure to prevent misuse and/or to prevent accidents). Since then, OpenAI has put out more recent content consistent with the latter view, and we are no longer aware of any clear disagreements. " Really, really happy to see this being carefully considered. Good job to the Open Philanthropy folks! EDIT: That Slate Star link is amazing: "Both sides here keep talking about who is going to “use” the superhuman intelligence a billion times more powerful than humanity, as if it were a microwave or something."
- cbr 10y agohttps://slatestarcodex.com/2015/12/17/should-ai-be-open/ https://slatestarcodex.com/2015/12/17/should-ai-be-open/
- throwaway667 10y agohush money?
- jonmc12 10y agoWhen OpenAI was announced, they mentioned having $1B in funding. Why the additional $30M?
- pjscott 10y agoSee section 2, "Case for the grant".
- MichaelGG 10y agoI went through it but didn't see anything addressing why OpenAI needed more money.
- forgotmysn 10y agowhere $1B is good, $1.03B is better?
- backpropaganda 10y agoThe $1B was just promised funding, not immediate. This $30M could be immediate. This makes me believe that perhaps the original funding is being dropped due to lack of results.
- WillyOnWheels 10y agoReading Bloomberg news too much makes me think AI research will only be used to classify ads and make more efficient securities trading algorithms. I will love to be proven wrong though.
- itchyjunk 10y agoIf some of the comments I read on other AI related articles here on HN are correct, 1 mil / year per expert * 10 experts per year = 30 Mil in 3 years Maybe $30 mil isn't as much as we think it is in AI business?
- sillysaurus3 10y ago1 mil / year per expert Is this realistic? I don't think it's impossible for a dev to be pulling $1M/yr in total comp, but it seemed more likely happen at Google or FB rather than AI.
- joshuamorton 10y agoNote that this says "per expert". I'm not sure what exactly you'd consider an "expert", but I think a reasonable definition of "expert" could result in your average expert being a million-dollar-a-year engineer.
- riffraff 10y agoDoes a thing like a million a year engineer exist? I feel like this number would make more sense for a pool of researchers with a strong lead than a single person.
- joshuamorton 10y agoWhile I obviously don't know for sure, I'm quite confident that there are a nonzero number of employees at any very large Tech Company that are engineers (ie. do development work, commit nontrivial code), and whose compensation is over $1,000,000 USD. I doubt its common, and I would expect that many of those employees are not "just" engineers (ie. at the point where you are doing work that is that valuable, its almost a certainty that you are leading a team, and you are designing things), but I'm confident they exist.
- mandor 10y ago... or more than 100 top French researchers for 3 years (Research Director, top pay at the end of the career is about 6000 euros / ~10000 for the company). [source : https://fr.wikipedia.org/wiki/Directeur_de_recherche_au_CNRS https://fr.wikipedia.org/wiki/Directeur_de_recherche_au_CNRS ]
- dilemma 10y agoTwo organizations that exploit the implications of the word "Open" as it is used in the world of technology to market their own private companies and organizations.
- deleted 10y ago[deleted]
- richardbatty 10y agoThe Open Philanthropy Project uses the word 'Open' to mean (http://www.openphilanthropy.org/what-open-means-us http://www.openphilanthropy.org/what-open-means-us): "Open to many possibilities ... instead of starting with a predefined set of focus areas, we’re considering a wide variety of causes where our philanthropy could help to improve others’ lives." and "Open about our work ... Very often, key discussions and decisions happen behind closed doors, and it’s difficult for outsiders to learn from and critique philanthropists’ work. We envision a world in which philanthropists increasingly document and share their research, reasoning, results and mistakes to help each other learn more quickly and serve others more effectively." This all seems pretty useful so I don't get what your criticism is.
- dilemma 10y agoMy criticism is that it isn't open, which is more or less absolute. Every organization is "a little" open, sharing the information that benefits them. And that seems to be what OpenAI intends to do: be open about that which suits them.
- deleted 10y ago[deleted]
- idlewords 10y agoScroll to the end. This is a $30M grant to the guy's roommate and future brother-in-law. Unbelievable.
- Analemma_ 10y agoidlewords, this comment has tremendous negative utility for humanity. By pointing out obvious nepotism and corruption in the AI risk charity field, you make people less likely to donate in the future, thus increasing the likelihood of a takeover from the robots. You must delete this comment; the fate of the world depends on it! /s
- nbouscal 10y agoTo the company that his roommate works for as a researcher. The way you're phrasing it is pretty misleading.
- idlewords 10y agoTwo of his roommates.
- deleted 10y ago[deleted]
- reachtarunhere 10y agoI don't see this as a problem. If I had tons of money and knew someone (by any connection) who was doing amazing work I would definitely chip in.
- hawkice 10y agoThere's lots of concern about the bizarre relationship disclosure. But perhaps even more bizarre is that this deal has a structure closer to a strategic move than actual philanthropy. Am I massively misreading this? This page details how their main goal with the $30M isn't to increase OpenAI's pledged funds by 3%, thereby reducing the marginal "AI Risk" by less than 3%. The goal is to have a seat on the board (basically -- they use a lot more words to say this in the announcement). What on earth is going on where a charitable organization with Open in its name feels it needs to buy its way onto the board of a prominent non-profit in order to: "Improve our understanding of the field of AI research" "[get] opportunities to become closely involved with any of the small number of existing organizations in “industry”" and "Better position us to generally promote the ideas and goals that we prioritize" Isn't the whole point of "open philanthropy" that you can direct funds to organizations more open about what's going on?!
- t3io 10y agoAssume for a minute that AGI is being developed and in no way shape or form does it function or is it formed in a manner that mainstream AI efforts focus on... That hypothetical could very well be the reality on the horizon. What of Safety/Control research that has fundamentally nothing to do with such a system or even its philosophy that the broad majority of these institutions or ventures are centered on? What of deep learning centric methodologies that are incompatible? Safety/control software and systems development isn't a research topic. It's an engineering practice that is most suited for well qualified and practiced engineers who design safety critical systems that are present all around you. Safety/Control Engineering isn't a 'lab experiment'. If one were aiming to secure, control and ensure the safety of a system, they'd likely hire a grey bearded team of engineers who are experts and have proven careers doing so. A particular systems design can be imparted on well qualified engineers. This happens everyday. Without a systems design or even a systems philosophy these efforts are just intellectual shots in the dark. Furthermore, has anyone even stopped to consider that these problems would get worked out naturally during the development of such a technology? Modern day AI algorithms and solutions center on mathematical optimization. AGI centers are far deeper and elusive constructs. One can ignore this all to clear truth all they like. So... If one's real concern is about the development of AGI and understanding therein, I think its fine time to admit that it might not come from the race horses everybody's betting on. As such, it is much more worth one's penny to start funding a diverse range of people and groups pursuing it who have sound ideas and solid approaches. This advice can continue to be ignored such as it currently is and has been for a number of years. It can persist across rather narrow hiring practices.... The closed/open door will or wont swing both ways.
- frik 10y agoCan someone explain why both orgs contain the world "Open" - I would say pretty misleading. OpenAI hasn't released any open code or anything open. And is OpenAI even about A.I.? (as several other here mentioned it's not AI)
- tlb 10y agoOpenAI has released several open-source packages (see https://github.com/openai https://github.com/openai and https://openai.com/systems/ https://openai.com/systems/) and several open research publications (see https://openai.com/research/ https://openai.com/research/)