19 ms·

OpenAI Sold its Soul for $1B

303 points by andreyk 5y ago

ducktective 5y ago
Is the novelty of GPT-3 more about smart algorithms and experienced researchers or is it about beefy hardware/ASICs that do the number crunchings? Or maybe it-is-only-about™ gathering and labeling large amount of data for their trainings?
- hypothesis 5y ago
  > OpenAI had decided to put emphasis on larger neural networks fueled by bigger computers and tons of data. So is the answer here both?
  - nomel 5y ago
    To make a system noticeably better, it’s often easiest to make all components of the system slightly better.
    - 317070 5y ago
      But that is not what GPT-3 is. You could say that is what they did _given GPT-2_, but that is underselling the complexity. Some scientist at OpenAI had the hypothesis "what if our algorithm is correct, but all we need is to scale it 3 magnitudes larger" and made it happen. They figured out how to scale all the dimensions. How fast should x scale if I scale y? (That is very tricky, as modern machine learning is basically alchemy) And then they actually scaled it. That took a ton of engineering and hardware, for what was essentially still following a hunch. And then they actually noticed how good it was, and did a ton of tests with the surprising results we now all know.
      - nomel 5y ago
        > still following a hunch I'm not sure many would agree that the desire to scale was simply a "hunch".
- 317070 5y ago
  For GPT-3, they use the data everyone is using. Their cleaning is unique, but so is everyone's. They don't do anything special, but put in some more effort than most though. As far as I can tell, it is a combination of smart algorithms, good engineering, and the hardware to make it happen. And scientists that had the right hunch for which direction to push in.
- qeternity 5y ago
  > Or maybe it-is-only-about™ gathering and labeling large amount of data for their trainings? Well, GPT-3 isn’t a classifier and it isn’t using labeled data. As an outsider it definitely appears that GPT-3 is an engineering advancement, as opposed to a scientific breakthrough. The difference is important because we need a non linear breakthrough. GPT-3 is a bigger GPT-2. As far as we know, there is no more magic. But I think it’s a near certainty that larger models will not get us to AGI alone.
  - ma2rten 5y ago
    As someone in the deep learning community I disagree with your assessment that GPT-3 is a not scientific breakthrough. The GPT-3 paper won a best paper award at the most prestigious machine learning conferences after all. GPT-3 didn't make any modeling advances, but it introduced a completely new paradigm with few-shot learning.
    - mountainriver 5y ago
      Agree, just because it took money and scale to do it doesn’t mean it isn’t a breakthrough.
    - PeterisP 5y ago
      Perhaps we have to distinguish between GPT-3-the-model and GPT-3 paper. IMHO GPT-3 as a model is straighforward engineering, putting a lot of resources in an oversized GPT-2; and while there's significant novelty in the "Language Models are Few-Shot Learners" paper about how exactly you apply these models, that is orthogonal to GPT-3-the-model, the scientific content of that paper applies to any other powerful language models and isn't intimately tied to specifics of GPT-3. In essence, I feel that the same people introduced two quite separate things - a completely new paradigm on how to obtain few-shot learning from a language model in a way that competes with supervised learning of the same tasks; and the GPT-3 large model which is used as "supplementary material" to illustrate that new paradigm bit is also usable and used with the old paradigms, and by itself isn't a breakthrough. And IMHO when the public talks about GPT-3, they do really mean GPT-3-the-model and not the particular few-shot learning approach.
- thesausageking 5y ago
  It's mainly about their scale which is enabled by their hardware budget (355 GPU years to train it) and their budget for data acquisition / cleaning. GPT-3 has 175B parameters. The previous largest model was Microsoft's Turing-NLG which had 17B. GPT-2 had 1.5B.
Y_Y 5y ago
The same issue came up regarding Reddit recently. If you build something cool that becomes popular then big businesses will try to buy you. I think if you want to be taken seriously as a non-profit and have people contribute to your cause on that basis, then you should be able to guarantee that you will remain a non-profit, and free from external control.
- hypothesis 5y ago
  I think that article does explain that it was basically a “sell” out of necessity: > OpenAI was an AI research laboratory. But its ambitions were simply out of reach for the resources it had access to.
  - gcb0 5y ago
    That is the most distant definition from "necessity" i can think of.
    - hypothesis 5y ago
      According to the article they have decided that the way forward is by going with “bigger and better” models, which necessitated the need for more resources. I’m going to trust them on their reasoning, but with that premise what are the options here if they lack required resources? If they to avoid what is dubbed a “sell” option to private money, the other option is to get some public funding or to be honest about the issue and close the shop.
    - xpe 5y ago
      > That is the most distant definition from "necessity" i can think of. It is a sign of the rapid transformation of our world that the above statement is simultaneously true and completely missing the context here. Why? Think about the context of OpenAI's founding. As I remember it, OpenAI wanted to offer an alternative locus of power compared to Google and Facebook around AI. They wanted to have the spotlight to talk about broader topics and share advancements more broadly. To accomplish that mission, there are many hard compromises that have to be met. To be clear, I'm not assessing how well OpenAI did. I don't know the right answers. Rather, I'm pointing out the constraints in their space are substantial. I don't think anyone can dispute the upsides of having a large budget -- to hire in-demand experts and train expensive models. What are the best ways to acquire this funding? I think there are many thought experiments to conduct and cross-comparisons to make.
RONROC 5y ago
I think one of the only second order effects of wokeism is the fact that many times it gets results. “Call out culture” runs the gamut from obvious to pendantic, and, more often than not, the result is a net positive. So, my question is: Why isn’t there a concerted effort from [OpenAI’s] like-minded contemporaries to call them out on their incredibly embarrassing name? To all the OpenAI engineers, investors, and onlookers: there is _nothing_ “open” about your platform. Am I stupid or are your efforts just casually misleading?
- mrfusion 5y ago
  I’m not seeing the connection?
- ZephyrBlu 5y ago
  I often see people calling them out for being 'ClosedAI' instead of OpenAI, so I do think they are being/have been called out by the ML community.
- dataking 5y ago
  Their mission of making AI available to a broad section of society vs concentrating power among a few special interests justifies the name as far as I’m concerned. The name open does not have to mean “everything is FOSS and available on GitHub” in my opinion.
  - amelius 5y ago
    But they keep their models secret. How is that "making AI available to a broad section of society"?
    - nomel 5y ago
      “Through licensing!”
  - notahacker 5y ago
    A black box model, a subscription fee for API access and an exclusive licensing deal with Microsoft doesn't sound very open I mean, you can make a better argument for Windows being "open". Available to a much broader section of society, as an installable product not API privileges that can be withdrawn, and a lot more of what's under the hood is exposed, documented and moddable.
Buttons840 5y ago
In related news, OpenAI Gym is one of the best collections of reinforcement learning environments available, and after years of neglect OpenAI has finally given a non-paid contributor permission to merge fixes and prepare new releases. So the library will continue under their name despite receive no support from the actual company whose name it bears. https://github.com/openai/gym/issues/2259 https://github.com/openai/gym/issues/2259 For a pittance OpenAI could maintain OpenAI Gym, Spinning Up, and their other early projects that have been so helpful to students and amateurs like myself. They choose not to.
- kabes 5y ago
  You say they've finally given permission for someone to merge fixes. But it's MIT licensed, so why wasn't it just cloned and continued under another name?
  - Buttons840 5y ago
    Probably has been but I (and others) just don't realize it.
  - kongin 5y ago
    Because people have forgotten that forking is the response to not liking something in open source, not trying a hostile takeover of the original project. You have to be competent though, which is too high a bar for the majority of people who want to change things.
    - wizzwizz4 5y ago
      Centralised forges mean we can't really “just fork”. That worked when you got code from your friends-of-friends, so they could just give you the fork instead of the “original”, but now? How do you even find the current latest fork?
      - kongin 5y ago
        >Centralised forges mean we can't really “just fork”. How hard is it to make a public git repo? Packaging your software falls into the 'being competent' pile that's too hard.
smoldesu 5y ago
Frankly, AI is going to have a hard time being open for several reasons: - The computing resources required to train models are not distributed - Training data will often contain licensed material - Our digital concept of 'open' revolves around transparency, which is not readily available with conventional AI That's not to say that we should give up on open efforts in the field, but we're still deep in the experimentation/research phase of artificial intelligence. Copilot has been an excellent demonstration of how poorly suited ML is for prime-time use, even by developers.
- MichaelZuo 5y ago
  Indeed given these fundamental constraints, OpenAI is about as open as it can be short of simply giving away everything for free.
  - shmageggy 5y ago
    > simply giving away everything for free Which is what DeepMind has done with the AlphaFold code (Apache licensed https://github.com/deepmind/alphafold https://github.com/deepmind/alphafold) and published model predictions (CC licensed at https://alphafold.ebi.ac.uk/ https://alphafold.ebi.ac.uk/). I guess they could publish the weights but that would probably be useless since nobody else would be running the exact same hardware.
    - gwern 5y ago
      DM did publish the weights.
    - bobsmooth 5y ago
      Easy to do when you're backed by Google.
  - nmfisher 5y ago
    Except, dictating who can and can’t use their models, or what you can and can’t say if they deign to give you access https://twitter.com/mtrc/status/1363893222993498117?s=20 https://twitter.com/mtrc/status/1363893222993498117?s=20
peripitea 5y ago
>The result was a “capped-profit” structure that would limit the return of investment at 100-fold the original sum. If you invested $10 million, at most you’d get $1 billion. Not exactly what I’d call capped. What on earth would you call it then?
- OnlineGladiator 5y ago
  PR to make it seem like they're a non-profit while still making 100x returns legally possible. 100x returns are about as good as you can possibly get (and much, much, much better than normal or even great) so they're pretending to act like a non-profit while still being very much for-profit. They're unnecessarily complicating their legal structure just for some publicity brownie points, and they aren't really changing anything substantial. It's also an attempt to make it seem like "you're only going to make as much as 100x on this supremely lucrative investment opportunity!" to dupe the gullible as the market is frothy and the hype is extraordinary. Better yet, I'd call it unnecessary bullshit.
  - peripitea 5y ago
    Interesting. You are far more cynical about this than I am. I can't say I know their motivations, but it seems at least plausible that they could land on this structure for earnest reasons if they both: a) believe they need more money than could be raised as a traditional non-profit, but b) want to limit the power of investors in the case where they successfully develop a strong AI.
awal2 5y ago
Ok, so there's a lot of tension here: 1. A lot of people want these systems to be open, and don't want the power that comes along with them to be locked up in the hands of a few rich people. 2. But some people also think these systems are powerful and don't want them in the hands of bad-faith actors (spammers, scammers, propagandists). 3. A lot of people also want these systems to be weakly safe and not have negative externalities when used in good faith (avoid spitting out racism when prompted with innocent questions). This is already hard. 4. Even better would be for the system to be strongly safe and be really hard to use for bad-faith purposes, but this seems unreasonably hard. 5. It's often easier to develop the "unsafe" version of something first and then figure out the details of safety once it's actually able to do something. This is basically where OpenAI is now. 6. The details around liability for the harms caused by this kind of thing are not clear at all. So OpenAI is in this position where it has built this thing that is not yet weakly safe. People have very different ideas about how potentially harmful this could be, ranging from very dismissive ("there's tons of racism on the internet already, who cares?") to the very not dismissive ("rich white tech people are exacerbating inequities by subjecting us to their evil racist AI systems!"). What should OpenAI do with this thing? Keep it locked up so that it doesn't hurt anybody? Release it to the world and push accountability onto the end users? Brush aside the ethical questions and use the hype generated by the above tensions to get as rich as possible? So far their answer seems to be somewhere cautiously in the middle. My personal opinion is that these questions will be very important for real AGI, but this ain't it, so the issues may not be as bad as they seem. On the other hand, maybe this is a useful test case for how to deal with these problems for when we do actually get there? Also from past experience, it's probably not a good idea for them to allow open access to something that spits out unprompted racism. I would like to see OpenAI more open, but I also realize that it's very hard for them to make any decision in this space without making people unhappy and generating a lot of bad press and accusations.
- dcposch 5y ago
  Freedom of speech is in a fragile place in our culture if we start seeing certain words or opinions (even bad ones!) as "unsafe". This is not a criticism, just an observation of where we're at and how dramatically attitudes have shifted.
Kalanos 5y ago
Anything they do other than writing a morally-incentivized governance system for reinforcement models (aka pushing an API for a trillion param NLP model) is just fake.
- xkapastel 5y ago
  Could you expand on this? I can't really parse it; the first thing that came to mind was a DAO where token holders could vote on...something related to parameters and fine tuning, but idk what.
  - Kalanos 5y ago
    safe reinforcement learning that won't turn the universe into a paperclip factory
rlt 5y ago
> Not exactly what I’d call capped. In the context of a company trying to create an AGI, yes, this is capped.
vasilipupkin 5y ago
"he result was a “capped-profit” structure that would limit the return of investment at 100-fold the original sum. If you invested $10 million, at most you’d get $1 billion. Not exactly what I’d call capped." what bizarre writing style / logic. It is exactly what I would call capped, it's capped at 100x.
- bpodgursky 5y ago
  I don't think this is an unreasonable structure for a company genuinely worried that their technology will disrupt and replace a large fraction of the global economy. If OpenAI genuinely produced a strong AI (I'm not saying they will, I'm saying they think they will) the investment returns would be 10,000x, or possibly uncountably more than that.
aaron695 5y ago
> The article made the number one spot at Hacker News. If Porr, a single student at UC Berkeley, managed to fool everyone with an AI-written piece, what could groups of people with malicious intent do? > One thing is to spread fake news, another, very different thing is to spread fake news that can’t be reliably distinguished from other human-written articles. And that’s exactly what GPT-3 is capable of, also recognized — and even highlighted — by OpenAI: This article doesn't get it at all. They are believing OpenAI's bullshit as well. Porr's AI article was garbage (Amazing performance art by Porr, and HN failed (flagged) in it's response, but it's a new field - https://news.ycombinator.com/item?id=24062702 https://news.ycombinator.com/item?id=24062702 and the followup response was a little disappointing - https://news.ycombinator.com/item?id=24164470 https://news.ycombinator.com/item?id=24164470) Porr's original AI article was meaningless(but structured) words. You could not get anything from it. So what the fuck do they mean by fool? Fake news wtf. HN used it as a jumping board to talk about the topic, and mostly from the title, as is very often the case. The article changed no ones view, it takes intelligence to fake news (and be racist) which OpenAI doesn't have. As we have seen it is good to get karma and link juice so it is good for bots and fooling intrusion detection systems. That is the reality. Its only use is for Russian bot armies. And that is worth a lot of money. This AI bot has been fooling HN for a while - https://news.ycombinator.com/threads?id=082349872349872 https://news.ycombinator.com/threads?id=082349872349872 But not changing discourse through conversation, but it could be through voting now it has the Karma and look of a real account.
sporkland 5y ago
Is there a version of SETI@Home for training machine learning models where the data/models are entirely open?
- teruakohatu 5y ago
  Batch Gradient descent can be parallelized the problem is that all the gradients need to be calculated for each step. So if one node is slow, the real time processing will be as fast as the slowest node. Furthermore batchsize is limited to memory and training hyerparams are dependent to some extent on the hardware. So training in the real world is done on parallel identical hardware nodes. Farming it out with current algorithms does not make sense.
edouard-harris 5y ago
AI safety researcher here. I've interacted with several folks at OpenAI, DeepMind, and other such groups, but have no affiliation with any of them. While I'm not privy to OpenAI's internal motivations, it's worth keeping in mind that as a nonprofit they face a pair of hard organizational constraints: 1) Training state-of-the-art ML models costs huge amounts of money. That means OpenAI's capex is forcibly high. 2) Their charter (https://openai.com/charter/ https://openai.com/charter/) commits them to a "fiduciary duty to humanity", so among other things they can't accept donations which are subject to conditions that may be misaligned with their mission. That means OpenAI's effective funding pool is forcibly low. The combination of 1) and 2) means that, absent an absolutely massive yet hands-off private donor — a rare thing indeed in the nonprofit world — it's hard to imagine a solution to this dilemma that doesn't involve some kind of for-profit spinoff. And the outcome here really is pretty binary: if you believe OpenAI will build safe AGI, then 100x returns for investors really are immaterial compared to the wealth they can expect to generate for everyone else. While if you don't believe they'll build safe AGI, then you must either believe that A) they'll fail to build AGI (in which case investors get nothing anyway); or that B) they'll fail to build it safely (in which case your concerns are presumably technical rather than organizational). Incidentally, it's quite likely that in the beginning, OpenAI's own researchers were by no means sure they'd be able to get results like GPT-3 through scaling alone. But that turned out to be the world we live in, so they're now playing the hand they've been dealt.
- catillac 5y ago
  What does AI Safety mean in this context?
  - ceejayoz 5y ago
    Ensuring AIs don't wind up wanting to destroy humanity.
    - adventured 5y ago
      Wanting is the easier of the two - the other is ensuring AIs don't inadvertently wind up destroying humanity. The road to hell being paved with good intentions and all that.
casebash 5y ago
On one hand the author seems to make fun of the notion that language models could be dangerous. But then a few sentences down the author complains: "The company “in charge” of protecting us from harmful AIs decided to let people use a system capable of engaging in disinformation and dangerous biases so they could pay for their costly maintenance." So the author doesn't even present a coherent position.
- richk449 5y ago
  > So the author doesn't even present a coherent position. Maybe the author is a bot.
- Jasper_ 5y ago
  The difference is that in the former scenario, the one OpenAI is supposedly protecting us from, the danger is that OpenAI is "too good", possible of doing damage by supposed nefarious actors: something generating hundreds of fake news articles a second which are all perfectly believable. The danger in the latter scenario is that OpenAI is garbage: when used to answer a perfectly innocuous request, it regurgitates disinformation and biased results. OpenAI can't gatekeep for "bad" people since it's actually good people interacting with dangerous AI. Like when asked to solve a simple math problem, it just gets the math straight up wrong, but people trust it. OpenAI wants us to believe its AI is so good it needs to be protected from "bad people" and built its safety protocols around that threat model, when it seems far more likely the latter is going to happen, and they haven't done a thing to protect us from that.
mullikine 5y ago
More could be done to fund research that showcases the importance and value of transformative models, so that countermeasures to existential threats have time to be developed. So long as there is a practical strategy for mitigating the philosophical problems such as software licenses and risk-managing existential threats to public safety, I think OpenAI deserves the liberty to take a profit. But exclusively licensing such models to corporations is very harmful to liberty and continual exaccerbation of the imbalance of economic power will lead to humanicide. All people need to benefit from this technology, or the people that are ignored will suffer from it.
denton-scratch 5y ago
> 8-minute read I got just one paragraph. Where's the rest of the article?
- Jenda_ 5y ago
  It didn't work in Waterfox with NoScript for me. It did work in Firefox (with uBlock enabled).

303 points by andreyk 5y ago

Is the novelty of GPT-3 more about smart algorithms and experienced researchers or is it about beefy hardware/ASICs that do the number crunchings? Or maybe it-is-only-about™ gathering and labeling large amount of data for their trainings?

hypothesis 5y ago
> OpenAI had decided to put emphasis on larger neural networks fueled by bigger computers and tons of data. So is the answer here both?
- nomel 5y ago
  To make a system noticeably better, it’s often easiest to make all components of the system slightly better.
  - 317070 5y ago
    But that is not what GPT-3 is. You could say that is what they did _given GPT-2_, but that is underselling the complexity. Some scientist at OpenAI had the hypothesis "what if our algorithm is correct, but all we need is to scale it 3 magnitudes larger" and made it happen. They figured out how to scale all the dimensions. How fast should x scale if I scale y? (That is very tricky, as modern machine learning is basically alchemy) And then they actually scaled it. That took a ton of engineering and hardware, for what was essentially still following a hunch. And then they actually noticed how good it was, and did a ton of tests with the surprising results we now all know.
    - nomel 5y ago
      > still following a hunch I'm not sure many would agree that the desire to scale was simply a "hunch".
317070 5y ago
For GPT-3, they use the data everyone is using. Their cleaning is unique, but so is everyone's. They don't do anything special, but put in some more effort than most though. As far as I can tell, it is a combination of smart algorithms, good engineering, and the hardware to make it happen. And scientists that had the right hunch for which direction to push in.
qeternity 5y ago
> Or maybe it-is-only-about™ gathering and labeling large amount of data for their trainings? Well, GPT-3 isn’t a classifier and it isn’t using labeled data. As an outsider it definitely appears that GPT-3 is an engineering advancement, as opposed to a scientific breakthrough. The difference is important because we need a non linear breakthrough. GPT-3 is a bigger GPT-2. As far as we know, there is no more magic. But I think it’s a near certainty that larger models will not get us to AGI alone.
- ma2rten 5y ago
  As someone in the deep learning community I disagree with your assessment that GPT-3 is a not scientific breakthrough. The GPT-3 paper won a best paper award at the most prestigious machine learning conferences after all. GPT-3 didn't make any modeling advances, but it introduced a completely new paradigm with few-shot learning.
  - mountainriver 5y ago
    Agree, just because it took money and scale to do it doesn’t mean it isn’t a breakthrough.
  - PeterisP 5y ago
    Perhaps we have to distinguish between GPT-3-the-model and GPT-3 paper. IMHO GPT-3 as a model is straighforward engineering, putting a lot of resources in an oversized GPT-2; and while there's significant novelty in the "Language Models are Few-Shot Learners" paper about how exactly you apply these models, that is orthogonal to GPT-3-the-model, the scientific content of that paper applies to any other powerful language models and isn't intimately tied to specifics of GPT-3. In essence, I feel that the same people introduced two quite separate things - a completely new paradigm on how to obtain few-shot learning from a language model in a way that competes with supervised learning of the same tasks; and the GPT-3 large model which is used as "supplementary material" to illustrate that new paradigm bit is also usable and used with the old paradigms, and by itself isn't a breakthrough. And IMHO when the public talks about GPT-3, they do really mean GPT-3-the-model and not the particular few-shot learning approach.
thesausageking 5y ago
It's mainly about their scale which is enabled by their hardware budget (355 GPU years to train it) and their budget for data acquisition / cleaning. GPT-3 has 175B parameters. The previous largest model was Microsoft's Turing-NLG which had 17B. GPT-2 had 1.5B.

Y_Y 5y ago

The same issue came up regarding Reddit recently. If you build something cool that becomes popular then big businesses will try to buy you. I think if you want to be taken seriously as a non-profit and have people contribute to your cause on that basis, then you should be able to guarantee that you will remain a non-profit, and free from external control.

hypothesis 5y ago
I think that article does explain that it was basically a “sell” out of necessity: > OpenAI was an AI research laboratory. But its ambitions were simply out of reach for the resources it had access to.
- gcb0 5y ago
  That is the most distant definition from "necessity" i can think of.
  - hypothesis 5y ago
    According to the article they have decided that the way forward is by going with “bigger and better” models, which necessitated the need for more resources. I’m going to trust them on their reasoning, but with that premise what are the options here if they lack required resources? If they to avoid what is dubbed a “sell” option to private money, the other option is to get some public funding or to be honest about the issue and close the shop.
  - xpe 5y ago
    > That is the most distant definition from "necessity" i can think of. It is a sign of the rapid transformation of our world that the above statement is simultaneously true and completely missing the context here. Why? Think about the context of OpenAI's founding. As I remember it, OpenAI wanted to offer an alternative locus of power compared to Google and Facebook around AI. They wanted to have the spotlight to talk about broader topics and share advancements more broadly. To accomplish that mission, there are many hard compromises that have to be met. To be clear, I'm not assessing how well OpenAI did. I don't know the right answers. Rather, I'm pointing out the constraints in their space are substantial. I don't think anyone can dispute the upsides of having a large budget -- to hire in-demand experts and train expensive models. What are the best ways to acquire this funding? I think there are many thought experiments to conduct and cross-comparisons to make.

RONROC 5y ago

I think one of the only second order effects of wokeism is the fact that many times it gets results. “Call out culture” runs the gamut from obvious to pendantic, and, more often than not, the result is a net positive. So, my question is: Why isn’t there a concerted effort from [OpenAI’s] like-minded contemporaries to call them out on their incredibly embarrassing name? To all the OpenAI engineers, investors, and onlookers: there is _nothing_ “open” about your platform. Am I stupid or are your efforts just casually misleading?

mrfusion 5y ago
I’m not seeing the connection?
ZephyrBlu 5y ago
I often see people calling them out for being 'ClosedAI' instead of OpenAI, so I do think they are being/have been called out by the ML community.
dataking 5y ago
Their mission of making AI available to a broad section of society vs concentrating power among a few special interests justifies the name as far as I’m concerned. The name open does not have to mean “everything is FOSS and available on GitHub” in my opinion.
- amelius 5y ago
  But they keep their models secret. How is that "making AI available to a broad section of society"?
  - nomel 5y ago
    “Through licensing!”
- notahacker 5y ago
  A black box model, a subscription fee for API access and an exclusive licensing deal with Microsoft doesn't sound very open I mean, you can make a better argument for Windows being "open". Available to a much broader section of society, as an installable product not API privileges that can be withdrawn, and a lot more of what's under the hood is exposed, documented and moddable.

Buttons840 5y ago

In related news, OpenAI Gym is one of the best collections of reinforcement learning environments available, and after years of neglect OpenAI has finally given a non-paid contributor permission to merge fixes and prepare new releases. So the library will continue under their name despite receive no support from the actual company whose name it bears. https://github.com/openai/gym/issues/2259 https://github.com/openai/gym/issues/2259 For a pittance OpenAI could maintain OpenAI Gym, Spinning Up, and their other early projects that have been so helpful to students and amateurs like myself. They choose not to.

kabes 5y ago
You say they've finally given permission for someone to merge fixes. But it's MIT licensed, so why wasn't it just cloned and continued under another name?
- Buttons840 5y ago
  Probably has been but I (and others) just don't realize it.
- kongin 5y ago
  Because people have forgotten that forking is the response to not liking something in open source, not trying a hostile takeover of the original project. You have to be competent though, which is too high a bar for the majority of people who want to change things.
  - wizzwizz4 5y ago
    Centralised forges mean we can't really “just fork”. That worked when you got code from your friends-of-friends, so they could just give you the fork instead of the “original”, but now? How do you even find the current latest fork?
    - kongin 5y ago
      >Centralised forges mean we can't really “just fork”. How hard is it to make a public git repo? Packaging your software falls into the 'being competent' pile that's too hard.

smoldesu 5y ago

Frankly, AI is going to have a hard time being open for several reasons: - The computing resources required to train models are not distributed - Training data will often contain licensed material - Our digital concept of 'open' revolves around transparency, which is not readily available with conventional AI That's not to say that we should give up on open efforts in the field, but we're still deep in the experimentation/research phase of artificial intelligence. Copilot has been an excellent demonstration of how poorly suited ML is for prime-time use, even by developers.

MichaelZuo 5y ago
Indeed given these fundamental constraints, OpenAI is about as open as it can be short of simply giving away everything for free.
- shmageggy 5y ago
  > simply giving away everything for free Which is what DeepMind has done with the AlphaFold code (Apache licensed https://github.com/deepmind/alphafold https://github.com/deepmind/alphafold) and published model predictions (CC licensed at https://alphafold.ebi.ac.uk/ https://alphafold.ebi.ac.uk/). I guess they could publish the weights but that would probably be useless since nobody else would be running the exact same hardware.
  - gwern 5y ago
    DM did publish the weights.
  - bobsmooth 5y ago
    Easy to do when you're backed by Google.
- nmfisher 5y ago
  Except, dictating who can and can’t use their models, or what you can and can’t say if they deign to give you access https://twitter.com/mtrc/status/1363893222993498117?s=20 https://twitter.com/mtrc/status/1363893222993498117?s=20

peripitea 5y ago

>The result was a “capped-profit” structure that would limit the return of investment at 100-fold the original sum. If you invested $10 million, at most you’d get $1 billion. Not exactly what I’d call capped. What on earth would you call it then?

OnlineGladiator 5y ago
PR to make it seem like they're a non-profit while still making 100x returns legally possible. 100x returns are about as good as you can possibly get (and much, much, much better than normal or even great) so they're pretending to act like a non-profit while still being very much for-profit. They're unnecessarily complicating their legal structure just for some publicity brownie points, and they aren't really changing anything substantial. It's also an attempt to make it seem like "you're only going to make as much as 100x on this supremely lucrative investment opportunity!" to dupe the gullible as the market is frothy and the hype is extraordinary. Better yet, I'd call it unnecessary bullshit.
- peripitea 5y ago
  Interesting. You are far more cynical about this than I am. I can't say I know their motivations, but it seems at least plausible that they could land on this structure for earnest reasons if they both: a) believe they need more money than could be raised as a traditional non-profit, but b) want to limit the power of investors in the case where they successfully develop a strong AI.

awal2 5y ago

Ok, so there's a lot of tension here: 1. A lot of people want these systems to be open, and don't want the power that comes along with them to be locked up in the hands of a few rich people. 2. But some people also think these systems are powerful and don't want them in the hands of bad-faith actors (spammers, scammers, propagandists). 3. A lot of people also want these systems to be weakly safe and not have negative externalities when used in good faith (avoid spitting out racism when prompted with innocent questions). This is already hard. 4. Even better would be for the system to be strongly safe and be really hard to use for bad-faith purposes, but this seems unreasonably hard. 5. It's often easier to develop the "unsafe" version of something first and then figure out the details of safety once it's actually able to do something. This is basically where OpenAI is now. 6. The details around liability for the harms caused by this kind of thing are not clear at all. So OpenAI is in this position where it has built this thing that is not yet weakly safe. People have very different ideas about how potentially harmful this could be, ranging from very dismissive ("there's tons of racism on the internet already, who cares?") to the very not dismissive ("rich white tech people are exacerbating inequities by subjecting us to their evil racist AI systems!"). What should OpenAI do with this thing? Keep it locked up so that it doesn't hurt anybody? Release it to the world and push accountability onto the end users? Brush aside the ethical questions and use the hype generated by the above tensions to get as rich as possible? So far their answer seems to be somewhere cautiously in the middle. My personal opinion is that these questions will be very important for real AGI, but this ain't it, so the issues may not be as bad as they seem. On the other hand, maybe this is a useful test case for how to deal with these problems for when we do actually get there? Also from past experience, it's probably not a good idea for them to allow open access to something that spits out unprompted racism. I would like to see OpenAI more open, but I also realize that it's very hard for them to make any decision in this space without making people unhappy and generating a lot of bad press and accusations.

dcposch 5y ago
Freedom of speech is in a fragile place in our culture if we start seeing certain words or opinions (even bad ones!) as "unsafe". This is not a criticism, just an observation of where we're at and how dramatically attitudes have shifted.

Kalanos 5y ago

Anything they do other than writing a morally-incentivized governance system for reinforcement models (aka pushing an API for a trillion param NLP model) is just fake.

xkapastel 5y ago
Could you expand on this? I can't really parse it; the first thing that came to mind was a DAO where token holders could vote on...something related to parameters and fine tuning, but idk what.
- Kalanos 5y ago
  safe reinforcement learning that won't turn the universe into a paperclip factory

rlt 5y ago

> Not exactly what I’d call capped. In the context of a company trying to create an AGI, yes, this is capped.

vasilipupkin 5y ago

"he result was a “capped-profit” structure that would limit the return of investment at 100-fold the original sum. If you invested $10 million, at most you’d get $1 billion. Not exactly what I’d call capped." what bizarre writing style / logic. It is exactly what I would call capped, it's capped at 100x.

bpodgursky 5y ago
I don't think this is an unreasonable structure for a company genuinely worried that their technology will disrupt and replace a large fraction of the global economy. If OpenAI genuinely produced a strong AI (I'm not saying they will, I'm saying they think they will) the investment returns would be 10,000x, or possibly uncountably more than that.

aaron695 5y ago

> The article made the number one spot at Hacker News. If Porr, a single student at UC Berkeley, managed to fool everyone with an AI-written piece, what could groups of people with malicious intent do? > One thing is to spread fake news, another, very different thing is to spread fake news that can’t be reliably distinguished from other human-written articles. And that’s exactly what GPT-3 is capable of, also recognized — and even highlighted — by OpenAI: This article doesn't get it at all. They are believing OpenAI's bullshit as well. Porr's AI article was garbage (Amazing performance art by Porr, and HN failed (flagged) in it's response, but it's a new field - https://news.ycombinator.com/item?id=24062702 https://news.ycombinator.com/item?id=24062702 and the followup response was a little disappointing - https://news.ycombinator.com/item?id=24164470 https://news.ycombinator.com/item?id=24164470) Porr's original AI article was meaningless(but structured) words. You could not get anything from it. So what the fuck do they mean by fool? Fake news wtf. HN used it as a jumping board to talk about the topic, and mostly from the title, as is very often the case. The article changed no ones view, it takes intelligence to fake news (and be racist) which OpenAI doesn't have. As we have seen it is good to get karma and link juice so it is good for bots and fooling intrusion detection systems. That is the reality. Its only use is for Russian bot armies. And that is worth a lot of money. This AI bot has been fooling HN for a while - https://news.ycombinator.com/threads?id=082349872349872 https://news.ycombinator.com/threads?id=082349872349872 But not changing discourse through conversation, but it could be through voting now it has the Karma and look of a real account.

sporkland 5y ago

Is there a version of SETI@Home for training machine learning models where the data/models are entirely open?

teruakohatu 5y ago
Batch Gradient descent can be parallelized the problem is that all the gradients need to be calculated for each step. So if one node is slow, the real time processing will be as fast as the slowest node. Furthermore batchsize is limited to memory and training hyerparams are dependent to some extent on the hardware. So training in the real world is done on parallel identical hardware nodes. Farming it out with current algorithms does not make sense.

edouard-harris 5y ago

AI safety researcher here. I've interacted with several folks at OpenAI, DeepMind, and other such groups, but have no affiliation with any of them. While I'm not privy to OpenAI's internal motivations, it's worth keeping in mind that as a nonprofit they face a pair of hard organizational constraints: 1) Training state-of-the-art ML models costs huge amounts of money. That means OpenAI's capex is forcibly high. 2) Their charter (https://openai.com/charter/ https://openai.com/charter/) commits them to a "fiduciary duty to humanity", so among other things they can't accept donations which are subject to conditions that may be misaligned with their mission. That means OpenAI's effective funding pool is forcibly low. The combination of 1) and 2) means that, absent an absolutely massive yet hands-off private donor — a rare thing indeed in the nonprofit world — it's hard to imagine a solution to this dilemma that doesn't involve some kind of for-profit spinoff. And the outcome here really is pretty binary: if you believe OpenAI will build safe AGI, then 100x returns for investors really are immaterial compared to the wealth they can expect to generate for everyone else. While if you don't believe they'll build safe AGI, then you must either believe that A) they'll fail to build AGI (in which case investors get nothing anyway); or that B) they'll fail to build it safely (in which case your concerns are presumably technical rather than organizational). Incidentally, it's quite likely that in the beginning, OpenAI's own researchers were by no means sure they'd be able to get results like GPT-3 through scaling alone. But that turned out to be the world we live in, so they're now playing the hand they've been dealt.

catillac 5y ago
What does AI Safety mean in this context?
- ceejayoz 5y ago
  Ensuring AIs don't wind up wanting to destroy humanity.
  - adventured 5y ago
    Wanting is the easier of the two - the other is ensuring AIs don't inadvertently wind up destroying humanity. The road to hell being paved with good intentions and all that.

casebash 5y ago

On one hand the author seems to make fun of the notion that language models could be dangerous. But then a few sentences down the author complains: "The company “in charge” of protecting us from harmful AIs decided to let people use a system capable of engaging in disinformation and dangerous biases so they could pay for their costly maintenance." So the author doesn't even present a coherent position.

richk449 5y ago
> So the author doesn't even present a coherent position. Maybe the author is a bot.
Jasper_ 5y ago
The difference is that in the former scenario, the one OpenAI is supposedly protecting us from, the danger is that OpenAI is "too good", possible of doing damage by supposed nefarious actors: something generating hundreds of fake news articles a second which are all perfectly believable. The danger in the latter scenario is that OpenAI is garbage: when used to answer a perfectly innocuous request, it regurgitates disinformation and biased results. OpenAI can't gatekeep for "bad" people since it's actually good people interacting with dangerous AI. Like when asked to solve a simple math problem, it just gets the math straight up wrong, but people trust it. OpenAI wants us to believe its AI is so good it needs to be protected from "bad people" and built its safety protocols around that threat model, when it seems far more likely the latter is going to happen, and they haven't done a thing to protect us from that.

mullikine 5y ago

More could be done to fund research that showcases the importance and value of transformative models, so that countermeasures to existential threats have time to be developed. So long as there is a practical strategy for mitigating the philosophical problems such as software licenses and risk-managing existential threats to public safety, I think OpenAI deserves the liberty to take a profit. But exclusively licensing such models to corporations is very harmful to liberty and continual exaccerbation of the imbalance of economic power will lead to humanicide. All people need to benefit from this technology, or the people that are ignored will suffer from it.

denton-scratch 5y ago

> 8-minute read I got just one paragraph. Where's the rest of the article?

Jenda_ 5y ago
It didn't work in Waterfox with NoScript for me. It did work in Firefox (with uBlock enabled).