6 ms·

Inside OpenAI

130 points by brtkbrtk 10y ago

freddealmeida 10y ago
I often wonder about this. Four things are needed to truly build advanced AI's (read deep learning, deep reinforcement learning): new algorithms, complex data sets, and advanced GPU based computing (optimally GPU in any case) but also an open community. I think AI research is one of the most open, and this openness is really at the center of its growth. So I am happy OpenAI has started since it is within this vein of sharing that the community has already built. But I certainly don't fully grasp how it can open AI to the world unless it can share rather valuable data sets (often impossible to get data such as personal health record), and make computation much much cheaper. Let me illustrate my concern; Alphago required not just 30m game sets and complex understanding of both a policy and value network design, but also 1000 CPU and 200+ GPU instances. Something on the order of a few million dollars to build and utilize. I look forward to the work coming from OpenAI. I hope it lives up to the hype. But I believe AI will more than likely remain squarely in large enterprise since the cost of developing applications will be high for the short and possibly for the near long term.
- tlb 10y ago
  I believe AI will more than likely remain squarely in large enterprise since the cost of developing applications will be high. For perspective, the same thing was said about search engines in 1995. At the time, server-class computers (big DEC and Sun boxes) were expensive. Altavista had a computer nobody could buy. But it turned out, with some cleverness you could use commodity PCs instead. NVidia is doing impressive work to bring the price/performance of hardware down. I think the vast majority of applications will be possible on a startup budget. I also think data sets will become more common and less critical over time. Unsupervised and reinforcement learning require less external data. And so much data is available openly.
  - vasilipupkin 10y ago
    "For perspective, the same thing was said about search engines in 1995" yeah, and since then, we've basically had 2 major search engines developed. Google and Bing :) and bing development cost billions
    - jsemrau 10y ago
      And even with all that investment Bing is not even working that well. It is good, but not on a Google level.
  - maaku 10y ago
    Do you realize how much computing power is behind google search?
    - taneq 10y ago
      I'd be interested in the amount of computing power per searchable web page. Sure, there's a huge amount of computing power behind Google's web services but I'd lay odds that it's much more efficient than it used to be. The internet is huge.
- AndrewKemendo 10y ago
  Four things are needed to truly build advanced AI's (read deep learning, deep reinforcement learning): new algorithms, complex data sets, and advanced GPU based computing (optimally GPU in any case) but also an open community. Actually we have no idea what the constituent parts of AGI are. What you mention are the current state of the art for narrow AI projects like classification and segmentation - which is basically 100% of machine/deep learning currently, but are not generalizable yet. As an example the pre-eminant biologically inspired computing researcher Richard Granger is skeptical (and I agree) that parallel silicon will be able to scale to the flexibility that we see in biological learning (aka General intelligence). Based on what I see so far from OpenAI I don't see them getting to AGI. They haven't stated it as an explicit goal, I think because they don't have a pathway (nobody does by the way).
  - studentrob 10y ago
    > Actually we have no idea what the constituent parts of AGI are. Yup. We don't even have a good test for knowing AGI. Frankly, we don't even know if we are AI or if everything is predestined. We don't know if true randomness exists. Currently no AI system can define its own goals. I'd like to know how Open AI would solve that. To me, Open AI seems like it will just end up as a giant open sourced machine learning toolset. That's great but not the initially stated goal, and they risk souring investors on future AI tech that actually has merit when they fail to achieve AGI.
    - AndrewKemendo 10y ago
      Yup. We don't even have a good test for knowing AGI. Very true and I think significantly overlooked. The closest thing I have ever found was the Universal Anytime Intelligence Test [1] [1] http://users.dsic.upv.es/proy/anynt/ http://users.dsic.upv.es/proy/anynt/
      - studentrob 10y ago
        That's a huge amount of links and information. Can you summarize?
- daveguy 10y ago
  Re:computation. I wonder if there are talks between OpenAI and BOINC to open up some of their computational resources. With the new OpenAI environments platform they include a peer review/validation process. That may help concerns that BOINC resources would be used commercially.
- doctorpangloss 10y ago
  > how it can open AI to the world unless it can share rather valuable data sets I think the spectacularly publishable performance from industry comes entirely from these massive datasets. AlphaGo needed way more energy and data than a person did to perform as well as it did. To a set of pedantic but extremely well-meaning researchers, that could be interpreted as a huge failure. Clearly whatever it's doing isn't the kind of intelligence that people have, since people don't need megawatts of power and 30 million game sets. But that's what big companies have that academic departments don't. Journals publish both approaches though, and we all win from great discoveries.
  - vhold 10y ago
    I think you're coming it at it from the perspective that making Lee Sedol was free? Imagine how many calories, all the fuel used to create those calories, his own direct transportation footprint, the time and money spent across creating him and his entire life. AlphaGo might not be relatively that inefficient.
    - sqeaky 10y ago
      Let's do some Math! Assuming 2000 Calories per day a megawatt is about 240.000 calories per second we can do some silly calculations. Assuming AlphaGo consumed one megawatt for 1 whole day then it consumed about 20,736,000,000. This is probably low. This is enough Calories for a human for 28,405 years. If we split the Calories accross 500 that accounts for 57 years apiece. I had no clue where this would turn out when I start. If we presume AlphaGo can run on a 1,000 Watt machine we can still divide all these numbers by 1,000 and they still hold, so 28.4 years of human calories.
stevesun21 10y ago
If it is time we, human being should imprint some logic laws into all AI projects which are intent to create unsupervised AI?
- jbpetersen 10y ago
  That's about as practical as making a computer capable of everything except intellectual property infringement.
mlinksva 10y ago
> OpenAI is not a charity. https://backchannel.com/how-elon-musk-and-y-combinator-plan-to-stop-computers-from-taking-over-17e0e27dd02a https://backchannel.com/how-elon-musk-and-y-combinator-plan-... includes quote from Musk: > And as a result of a number of conversations, we came to the conclusion that having a 501c3, a non-profit, with no obligation to maximize profitability, would probably be a good thing to do. A 501c3 is a charity. Is "not a charity" inaccurate, or did OpenAI decide on some other organizational form, perhaps a trade association?
- studentrob 10y ago
  This may help: Public Charity vs. Private Foundation [1] Generally, I think charities are understood to use most of their funds to directly support the people in need rather than spending money on administration of the charity. By the way, non-profits are not viewed as inherently good by those within the sector, > Charity non-profits face many of the same challenges of corporate governance which face large, publicly traded corporations. Fundamentally, the challenges arise from the "agency problem" - the fact that the management which controls the charity is necessarily different from the people who the charity is designed to benefit [2] [1] https://www.501c3.org/public-charity-vs-private-foundation/ https://www.501c3.org/public-charity-vs-private-foundation/ [2] https://en.wikipedia.org/wiki/United_States_non-profit_laws#Organization https://en.wikipedia.org/wiki/United_States_non-profit_laws#...

130 points by brtkbrtk 10y ago

I often wonder about this. Four things are needed to truly build advanced AI's (read deep learning, deep reinforcement learning): new algorithms, complex data sets, and advanced GPU based computing (optimally GPU in any case) but also an open community. I think AI research is one of the most open, and this openness is really at the center of its growth. So I am happy OpenAI has started since it is within this vein of sharing that the community has already built. But I certainly don't fully grasp how it can open AI to the world unless it can share rather valuable data sets (often impossible to get data such as personal health record), and make computation much much cheaper. Let me illustrate my concern; Alphago required not just 30m game sets and complex understanding of both a policy and value network design, but also 1000 CPU and 200+ GPU instances. Something on the order of a few million dollars to build and utilize. I look forward to the work coming from OpenAI. I hope it lives up to the hype. But I believe AI will more than likely remain squarely in large enterprise since the cost of developing applications will be high for the short and possibly for the near long term.

tlb 10y ago
I believe AI will more than likely remain squarely in large enterprise since the cost of developing applications will be high. For perspective, the same thing was said about search engines in 1995. At the time, server-class computers (big DEC and Sun boxes) were expensive. Altavista had a computer nobody could buy. But it turned out, with some cleverness you could use commodity PCs instead. NVidia is doing impressive work to bring the price/performance of hardware down. I think the vast majority of applications will be possible on a startup budget. I also think data sets will become more common and less critical over time. Unsupervised and reinforcement learning require less external data. And so much data is available openly.
- vasilipupkin 10y ago
  "For perspective, the same thing was said about search engines in 1995" yeah, and since then, we've basically had 2 major search engines developed. Google and Bing :) and bing development cost billions
  - jsemrau 10y ago
    And even with all that investment Bing is not even working that well. It is good, but not on a Google level.
- maaku 10y ago
  Do you realize how much computing power is behind google search?
  - taneq 10y ago
    I'd be interested in the amount of computing power per searchable web page. Sure, there's a huge amount of computing power behind Google's web services but I'd lay odds that it's much more efficient than it used to be. The internet is huge.
AndrewKemendo 10y ago
Four things are needed to truly build advanced AI's (read deep learning, deep reinforcement learning): new algorithms, complex data sets, and advanced GPU based computing (optimally GPU in any case) but also an open community. Actually we have no idea what the constituent parts of AGI are. What you mention are the current state of the art for narrow AI projects like classification and segmentation - which is basically 100% of machine/deep learning currently, but are not generalizable yet. As an example the pre-eminant biologically inspired computing researcher Richard Granger is skeptical (and I agree) that parallel silicon will be able to scale to the flexibility that we see in biological learning (aka General intelligence). Based on what I see so far from OpenAI I don't see them getting to AGI. They haven't stated it as an explicit goal, I think because they don't have a pathway (nobody does by the way).
- studentrob 10y ago
  > Actually we have no idea what the constituent parts of AGI are. Yup. We don't even have a good test for knowing AGI. Frankly, we don't even know if we are AI or if everything is predestined. We don't know if true randomness exists. Currently no AI system can define its own goals. I'd like to know how Open AI would solve that. To me, Open AI seems like it will just end up as a giant open sourced machine learning toolset. That's great but not the initially stated goal, and they risk souring investors on future AI tech that actually has merit when they fail to achieve AGI.
  - AndrewKemendo 10y ago
    Yup. We don't even have a good test for knowing AGI. Very true and I think significantly overlooked. The closest thing I have ever found was the Universal Anytime Intelligence Test [1] [1] http://users.dsic.upv.es/proy/anynt/ http://users.dsic.upv.es/proy/anynt/
    - studentrob 10y ago
      That's a huge amount of links and information. Can you summarize?
daveguy 10y ago
Re:computation. I wonder if there are talks between OpenAI and BOINC to open up some of their computational resources. With the new OpenAI environments platform they include a peer review/validation process. That may help concerns that BOINC resources would be used commercially.
doctorpangloss 10y ago
> how it can open AI to the world unless it can share rather valuable data sets I think the spectacularly publishable performance from industry comes entirely from these massive datasets. AlphaGo needed way more energy and data than a person did to perform as well as it did. To a set of pedantic but extremely well-meaning researchers, that could be interpreted as a huge failure. Clearly whatever it's doing isn't the kind of intelligence that people have, since people don't need megawatts of power and 30 million game sets. But that's what big companies have that academic departments don't. Journals publish both approaches though, and we all win from great discoveries.
- vhold 10y ago
  I think you're coming it at it from the perspective that making Lee Sedol was free? Imagine how many calories, all the fuel used to create those calories, his own direct transportation footprint, the time and money spent across creating him and his entire life. AlphaGo might not be relatively that inefficient.
  - sqeaky 10y ago
    Let's do some Math! Assuming 2000 Calories per day a megawatt is about 240.000 calories per second we can do some silly calculations. Assuming AlphaGo consumed one megawatt for 1 whole day then it consumed about 20,736,000,000. This is probably low. This is enough Calories for a human for 28,405 years. If we split the Calories accross 500 that accounts for 57 years apiece. I had no clue where this would turn out when I start. If we presume AlphaGo can run on a 1,000 Watt machine we can still divide all these numbers by 1,000 and they still hold, so 28.4 years of human calories.

stevesun21 10y ago

If it is time we, human being should imprint some logic laws into all AI projects which are intent to create unsupervised AI?

jbpetersen 10y ago
That's about as practical as making a computer capable of everything except intellectual property infringement.

mlinksva 10y ago

> OpenAI is not a charity. https://backchannel.com/how-elon-musk-and-y-combinator-plan-to-stop-computers-from-taking-over-17e0e27dd02a https://backchannel.com/how-elon-musk-and-y-combinator-plan-... includes quote from Musk: > And as a result of a number of conversations, we came to the conclusion that having a 501c3, a non-profit, with no obligation to maximize profitability, would probably be a good thing to do. A 501c3 is a charity. Is "not a charity" inaccurate, or did OpenAI decide on some other organizational form, perhaps a trade association?

studentrob 10y ago
This may help: Public Charity vs. Private Foundation [1] Generally, I think charities are understood to use most of their funds to directly support the people in need rather than spending money on administration of the charity. By the way, non-profits are not viewed as inherently good by those within the sector, > Charity non-profits face many of the same challenges of corporate governance which face large, publicly traded corporations. Fundamentally, the challenges arise from the "agency problem" - the fact that the management which controls the charity is necessarily different from the people who the charity is designed to benefit [2] [1] https://www.501c3.org/public-charity-vs-private-foundation/ https://www.501c3.org/public-charity-vs-private-foundation/ [2] https://en.wikipedia.org/wiki/United_States_non-profit_laws#Organization https://en.wikipedia.org/wiki/United_States_non-profit_laws#...