7 ms·

Wow. Screw non-profit, we want to get rich. Sorry guys, but before you were probably able to get talent which is not (primarily) motivated by money. Now you ar

by jpdus 8y ago

Wow. Screw non-profit, we want to get rich. Sorry guys, but before you were probably able to get talent which is not (primarily) motivated by money. Now you are just another AI startup. If the cap would be 2x, it could still make sense. But 100x times? That's laughable! And the split board, made up of friends and closely connected people smells like "greenwashing" as well. Don't get me wrong, it's totally ok to be an AI startup. You just shouldn't pretend to be a non-profit then...

Judgmentality 8y ago
I agree, this sounds disappointing to me as well. My issue is how they're positioning themselves, which is basically a hyper-growth startup where you can get rich (but only 100x richer, because we're not greedy like other startups) but we're also a non-profit here to benefit humanity so don't tax us like those evil corporations. What really bothers me though is I don't know if they honestly believe what they're saying or if it's just a marketing ploy, because honestly it's so much worse if they're deluding themselves.
- gdb 8y ago
  Any returns from OpenAI LP are subject to taxes!
  - Judgmentality 8y ago
    First of all, thank you for taking the time to respond to people criticizing this announcement. I just feel like you're trying to have the best of both worlds. You want the hypergrowth startup that attracts talent and investors, but you also want the mission statement for people that aren't motivated by money. I suspect trying to maintain this middle ground will be an incredibly damaging factor moving forward, as the people who are purely profit driven will look elsewhere and the people who are truly mission driven will also look elsewhere. I fully appreciate how challenging these things can be and that making these decisions aren't trivial and obviously you're trying to do something different...but I really think you're taking the wrong step here. But because you will see short term gains from all the extra initial investment you won't realize it for years until it's too late and the culture has permanently shifted. That said, while my trust has somewhat eroded (also because you wouldn't release details of your model recently), I still wish you luck in your mission.
    - gdb 8y ago
      Thanks for the thoughtful reply, it's much appreciated. > you also want the mission statement for people that aren't motivated by money I wouldn't agree with this — we want people who are motivated by AGI going well, and don't want to capture all of its unprecedentedly large value for themselves. We think it's a strong point that OpenAI LP aligns individuals' success with success of the mission (and if the two conflict, the mission wins). We also think it's very important not to release technology we think might be harmful, as we wrote in the Charter: https://openai.com/charter/#cooperativeorientation https://openai.com/charter/#cooperativeorientation. There was a polarized response, but I'd rather err on the side of caution. Would love people who think like that to apply: https://openai.com/jobs https://openai.com/jobs
gdb 8y ago
(I work at OpenAI.) I think this tweet from one of our employees sums it up well: https://twitter.com/Miles_Brundage/status/1105190434052005889 https://twitter.com/Miles_Brundage/status/110519043405200588... Why are we making this move? Our mission is to ensure AGI benefits all of humanity, and our primary approach to doing this is to actually try building safe AGI. We need to raise billions of dollars to do this, and needed a structure like OpenAI LP to attract that kind of investment while staying true to the mission. If we succeed, the return will be exceed the cap by orders of magnitude. See https://blog.gregbrockman.com/the-openai-mission https://blog.gregbrockman.com/the-openai-mission for more details on how we think about the mission.
- nck4222 8y ago
  I believe you. I also believe there are now going to be outside parties with strong financial incentives in OpenAI who are not altruistic. I also believe this new structure will attract employees with less altruistic goals, that could slowly change the culture of OpenAI. I also believe there's nothing stopping anyone from changing the OpenAI mission further over time, other than the culture, which is now more susceptible to change.
  - nojvek 8y ago
    Something something money and power corrupts? We can just look at Google and see that “do no evil” does not work when you’ve got billions of dollars and reach into everyone’s private lives.
- jackpirate 8y ago
  If we succeed, the return will be exceed the cap by orders of magnitude. Are there any concrete estimates of the economic return that different levels of AGI would generate? It's not immediately obvious to me that an AGI would be worth more than 10 trillion dollars (which I believe is what would be needed for your claim to be true). For example, if there really is an "AGI algorithm", then what's to stop your competitors from implementing it too? Trends in ML research have shown that for most advances, there have been several groups working on similar projects independently at the same time, so other groups would likely be able to implement your AGI algorithm pretty easily even if you don't publish the details. And these competitors will drive down the profit you can expect from the AGI algorithm. If the trick really is the huge datasets/compute (which your recent results seem to suggest), then it may turn out that the power needed to run an AGI costs more than the potential return that the AGI can generate.
wycs 8y ago
I don't see the problem. If they get AGI, it will create value much larger than 100 billion. Much larger than trillions to be honest. If they fail to create AGI, then who cares?
- codekilla 8y ago
  > (AGI) — which we define as automated systems that outperform humans at most economically valuable work — [0] I don't doubt that OpenAI will be doing absolute first class AI research (they are already doing this). It's just that I don't really find this definition of 'GI' compelling, and 'Artificial' really doesn't mean much--just because you didn't find it in a meadow somewhere doesn't mean it doesn't work. So 'A' is a pointless qualification in my opinion. For me, the important part is what you define 'GI' to be, and I don't like the given definition. What we will have is world class task automation--which is going to be insanely profitable (congrats). But I would prefer not to confuse the idea with HLI(human-level intelligence). See [1] for a good discussion. They will fail to create AGI--mainly because we have no measurable definition of it. What they care about is how dangerous these systems could potentially be. More than nukes? It doesn't actually matter, who will stop who from using nukes or AGI or a superbug? Only political systems and worldwide cooperation can effectively deal with this...not a startup...not now...not ever. Period. [0] https://blog.gregbrockman.com/the-openai-mission https://blog.gregbrockman.com/the-openai-mission [1] https://dl.acm.org/citation.cfm?id=3281635.3271625&coll=portal&dl=ACM https://dl.acm.org/citation.cfm?id=3281635.3271625&coll=port...
  - deleted 8y ago
    [deleted]
ilyasut 8y ago
We have to raise a lot money to get a lot of compute, so we've created the best structure possible that will allow us to do so while maintaining maximal adherence to our mission. And if we actually succeed in building the safe AGI, we will generate far more value than any existing company, which will make the 100x cap very relevant.
- m_ke 8y ago
  What makes you believe that you'll get there first?
  - not_ai_yes_pr 8y ago
    I share the sentiment with you. Majority of the 'research' from OpenAI has been scaling up the known algorithms and almost all the models have been built on top of research from outside of OpenAI. My assessment is that OpenAI is currently not the leader in the field but they want to get there by attracting talent through PR and money, which IMHO is a fine strategy.
- codekilla 8y ago
  Why not open this compute up to the greater scientific community? We could use it, not just for AI.
- not_ai_yes_pr 8y ago
  What makes you think AGI is even possible? Most of current 'AI' is pattern recognition/pattern generation. I'm skeptical about the claims of AGI even being possible but I am confident that pattern recognition will be tremendously useful.
  - brian_cloutier 8y ago
    What makes you so sure that what you're doing isn't pattern recognition? When you lean a language, aren't you just matching sounds with the contexts in which they're used? What does "love" mean? 10 different people would probably give you 10 different answers, and few of them would mention that the way you love your apple is pretty distinct from the way you love your spouse. Though, even though they failed to mention it, they wouldn't misunderstand you when you did mention loving some apple! And it's not just vocabulary, the successes of RNNs show that grammar is also mostly patterns. Complicated and hard to describe patterns, for sure, but the RNN learns it can't say "the ball run" in just the same way you learn to say "the ball runs", by seeing enough examples that some constructions just sound right and some sound wrong. If you hadn't heard of AlphaGo you probably wouldn't agree that Go was "just" pattern matching. There's tactics, strategy(!), surely it's more than just looking at a board and deciding which moves feel right. And the articles about how chess masters "only see good moves"? Probably not related, right? What does your expensive database consultant do? Do they really do anything more than looking at some charts and matching those against problems they've seen before? Are you sure? https://blog.acolyer.org/2017/08/11/automatic-database-management-system-tuning-through-large-scale-machine-learning https://blog.acolyer.org/2017/08/11/automatic-database-manag...
deleted 8y ago
[deleted]

by jpdus 8y ago

Judgmentality 8y ago

I agree, this sounds disappointing to me as well. My issue is how they're positioning themselves, which is basically a hyper-growth startup where you can get rich (but only 100x richer, because we're not greedy like other startups) but we're also a non-profit here to benefit humanity so don't tax us like those evil corporations. What really bothers me though is I don't know if they honestly believe what they're saying or if it's just a marketing ploy, because honestly it's so much worse if they're deluding themselves.

gdb 8y ago
Any returns from OpenAI LP are subject to taxes!
- Judgmentality 8y ago
  First of all, thank you for taking the time to respond to people criticizing this announcement. I just feel like you're trying to have the best of both worlds. You want the hypergrowth startup that attracts talent and investors, but you also want the mission statement for people that aren't motivated by money. I suspect trying to maintain this middle ground will be an incredibly damaging factor moving forward, as the people who are purely profit driven will look elsewhere and the people who are truly mission driven will also look elsewhere. I fully appreciate how challenging these things can be and that making these decisions aren't trivial and obviously you're trying to do something different...but I really think you're taking the wrong step here. But because you will see short term gains from all the extra initial investment you won't realize it for years until it's too late and the culture has permanently shifted. That said, while my trust has somewhat eroded (also because you wouldn't release details of your model recently), I still wish you luck in your mission.
  - gdb 8y ago
    Thanks for the thoughtful reply, it's much appreciated. > you also want the mission statement for people that aren't motivated by money I wouldn't agree with this — we want people who are motivated by AGI going well, and don't want to capture all of its unprecedentedly large value for themselves. We think it's a strong point that OpenAI LP aligns individuals' success with success of the mission (and if the two conflict, the mission wins). We also think it's very important not to release technology we think might be harmful, as we wrote in the Charter: https://openai.com/charter/#cooperativeorientation https://openai.com/charter/#cooperativeorientation. There was a polarized response, but I'd rather err on the side of caution. Would love people who think like that to apply: https://openai.com/jobs https://openai.com/jobs

gdb 8y ago

(I work at OpenAI.) I think this tweet from one of our employees sums it up well: https://twitter.com/Miles_Brundage/status/1105190434052005889 https://twitter.com/Miles_Brundage/status/110519043405200588... Why are we making this move? Our mission is to ensure AGI benefits all of humanity, and our primary approach to doing this is to actually try building safe AGI. We need to raise billions of dollars to do this, and needed a structure like OpenAI LP to attract that kind of investment while staying true to the mission. If we succeed, the return will be exceed the cap by orders of magnitude. See https://blog.gregbrockman.com/the-openai-mission https://blog.gregbrockman.com/the-openai-mission for more details on how we think about the mission.

nck4222 8y ago
I believe you. I also believe there are now going to be outside parties with strong financial incentives in OpenAI who are not altruistic. I also believe this new structure will attract employees with less altruistic goals, that could slowly change the culture of OpenAI. I also believe there's nothing stopping anyone from changing the OpenAI mission further over time, other than the culture, which is now more susceptible to change.
- nojvek 8y ago
  Something something money and power corrupts? We can just look at Google and see that “do no evil” does not work when you’ve got billions of dollars and reach into everyone’s private lives.
jackpirate 8y ago
If we succeed, the return will be exceed the cap by orders of magnitude. Are there any concrete estimates of the economic return that different levels of AGI would generate? It's not immediately obvious to me that an AGI would be worth more than 10 trillion dollars (which I believe is what would be needed for your claim to be true). For example, if there really is an "AGI algorithm", then what's to stop your competitors from implementing it too? Trends in ML research have shown that for most advances, there have been several groups working on similar projects independently at the same time, so other groups would likely be able to implement your AGI algorithm pretty easily even if you don't publish the details. And these competitors will drive down the profit you can expect from the AGI algorithm. If the trick really is the huge datasets/compute (which your recent results seem to suggest), then it may turn out that the power needed to run an AGI costs more than the potential return that the AGI can generate.

wycs 8y ago

I don't see the problem. If they get AGI, it will create value much larger than 100 billion. Much larger than trillions to be honest. If they fail to create AGI, then who cares?

codekilla 8y ago
> (AGI) — which we define as automated systems that outperform humans at most economically valuable work — [0] I don't doubt that OpenAI will be doing absolute first class AI research (they are already doing this). It's just that I don't really find this definition of 'GI' compelling, and 'Artificial' really doesn't mean much--just because you didn't find it in a meadow somewhere doesn't mean it doesn't work. So 'A' is a pointless qualification in my opinion. For me, the important part is what you define 'GI' to be, and I don't like the given definition. What we will have is world class task automation--which is going to be insanely profitable (congrats). But I would prefer not to confuse the idea with HLI(human-level intelligence). See [1] for a good discussion. They will fail to create AGI--mainly because we have no measurable definition of it. What they care about is how dangerous these systems could potentially be. More than nukes? It doesn't actually matter, who will stop who from using nukes or AGI or a superbug? Only political systems and worldwide cooperation can effectively deal with this...not a startup...not now...not ever. Period. [0] https://blog.gregbrockman.com/the-openai-mission https://blog.gregbrockman.com/the-openai-mission [1] https://dl.acm.org/citation.cfm?id=3281635.3271625&coll=portal&dl=ACM https://dl.acm.org/citation.cfm?id=3281635.3271625&coll=port...
- deleted 8y ago
  [deleted]

ilyasut 8y ago

We have to raise a lot money to get a lot of compute, so we've created the best structure possible that will allow us to do so while maintaining maximal adherence to our mission. And if we actually succeed in building the safe AGI, we will generate far more value than any existing company, which will make the 100x cap very relevant.

m_ke 8y ago
What makes you believe that you'll get there first?
- not_ai_yes_pr 8y ago
  I share the sentiment with you. Majority of the 'research' from OpenAI has been scaling up the known algorithms and almost all the models have been built on top of research from outside of OpenAI. My assessment is that OpenAI is currently not the leader in the field but they want to get there by attracting talent through PR and money, which IMHO is a fine strategy.
codekilla 8y ago
Why not open this compute up to the greater scientific community? We could use it, not just for AI.
not_ai_yes_pr 8y ago
What makes you think AGI is even possible? Most of current 'AI' is pattern recognition/pattern generation. I'm skeptical about the claims of AGI even being possible but I am confident that pattern recognition will be tremendously useful.
- brian_cloutier 8y ago
  What makes you so sure that what you're doing isn't pattern recognition? When you lean a language, aren't you just matching sounds with the contexts in which they're used? What does "love" mean? 10 different people would probably give you 10 different answers, and few of them would mention that the way you love your apple is pretty distinct from the way you love your spouse. Though, even though they failed to mention it, they wouldn't misunderstand you when you did mention loving some apple! And it's not just vocabulary, the successes of RNNs show that grammar is also mostly patterns. Complicated and hard to describe patterns, for sure, but the RNN learns it can't say "the ball run" in just the same way you learn to say "the ball runs", by seeing enough examples that some constructions just sound right and some sound wrong. If you hadn't heard of AlphaGo you probably wouldn't agree that Go was "just" pattern matching. There's tactics, strategy(!), surely it's more than just looking at a board and deciding which moves feel right. And the articles about how chess masters "only see good moves"? Probably not related, right? What does your expensive database consultant do? Do they really do anything more than looking at some charts and matching those against problems they've seen before? Are you sure? https://blog.acolyer.org/2017/08/11/automatic-database-management-system-tuning-through-large-scale-machine-learning https://blog.acolyer.org/2017/08/11/automatic-database-manag...

deleted 8y ago

[deleted]