10 ms·

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `

by gwd 4y ago

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible. An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive / reference for this type of purpose. Open-sourcing older models shouldn't result in too much of a risk, either from an "AI Safety" perspective, or from a competitive perspective.

danpalmer 4y ago
And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.
- gwd 4y ago
  I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? Obviously that's going to affect your opinions about things. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions.
  - Dalewyn 4y ago
    >Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals).
    - gwd 4y ago
      > There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model. > In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals). Right, and that's why I said "would if they could". From their perspective, saving the human race would require stopping all research, including research done in China.
      - oefnak 4y ago
        You sound rational. Do you not agree with the possibility of AI doom soon?
        deleted 4y ago
        [deleted]
        sverona 4y ago
        It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
phkahler 4y ago
>> An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive Another obvious thing to do is do your research on non-commercial or open source things that can not be taken away from you. Sorry, I don't mean for the snark present in that statement. The frustration lies with the company and others that tend to pull rugs.
- UncleEntity 4y ago
  Exactly, expecting a company to maintain a project forever (or give it away) just because they were used in research projects is ludicrous. Maybe they did the research when “open” in their name meant something but it has been obvious for quite a while that ship has sailed.
  - red1reaper 4y ago
    Expecting a company to maintain a project forever or give it away just because they were used in research projects is actually ver reasonable.
    - UncleEntity 4y ago
      Very reasonable if you’re the one doing the research but not if you’re the one subsidizing the maintenance of an obsolete project. Open sourcing it is its own can of worms because they may be using third-party code they can’t release or, you know, &etc.
- z3c0 4y ago
  I thought I must be going crazy until I saw your comment. This sounds like a bad research practice that probably shouldn't be reproduced to begin with.
  - itsaquicknote 4y ago
    Research into systemically important infrastructure cannot be damned because that infrastructure isn't public. It's a cheap moralizing argument to say "pfff, this was predictable". Maybe so, but there isn't an alternative. Much like research on Twitter. Once these companies start to drift into providing what become broadscale social utilities and public services it doesn't matter that they're private. There are(/should be) obligations that come with that. You can't handwave and say go do your research on some micro-niche open source project that's way behind the SOTA and has nowhere near the same reach. That's not what "best practice" means here.
drexlspivey 4y ago
> we didn’t realize how important code-davinci-002 was to researchers, so we are keeping it going in our researcher access program: https://openai.com/form/researcher-access-program https://openai.com/form/researcher-access-program > we are also providing researcher access to the base GPT-4 model! https://twitter.com/sama/status/1638576434485825536 https://twitter.com/sama/status/1638576434485825536
varispeed 4y ago
The question is why would you start writing a paper based on a model you don't have control over? Surely the right way to do it, is to get your university to fund a creation of such a model first and then do research? Seems like researchers didn't think this through.
- Xelynega 4y ago
  Do research on the state of the art technology, or a homegrown copy of it that likely doesn't exhibit the same features. It seems like they thought it through to me.
deleted 4y ago
[deleted]

by gwd 4y ago

danpalmer 4y ago

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

gwd 4y ago
I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? Obviously that's going to affect your opinions about things. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions.
- Dalewyn 4y ago
  >Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals).
  - gwd 4y ago
    > There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model. > In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals). Right, and that's why I said "would if they could". From their perspective, saving the human race would require stopping all research, including research done in China.
    - oefnak 4y ago
      You sound rational. Do you not agree with the possibility of AI doom soon?
      - deleted 4y ago
        [deleted]
      - sverona 4y ago
        It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

phkahler 4y ago

>> An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive Another obvious thing to do is do your research on non-commercial or open source things that can not be taken away from you. Sorry, I don't mean for the snark present in that statement. The frustration lies with the company and others that tend to pull rugs.

UncleEntity 4y ago
Exactly, expecting a company to maintain a project forever (or give it away) just because they were used in research projects is ludicrous. Maybe they did the research when “open” in their name meant something but it has been obvious for quite a while that ship has sailed.
- red1reaper 4y ago
  Expecting a company to maintain a project forever or give it away just because they were used in research projects is actually ver reasonable.
  - UncleEntity 4y ago
    Very reasonable if you’re the one doing the research but not if you’re the one subsidizing the maintenance of an obsolete project. Open sourcing it is its own can of worms because they may be using third-party code they can’t release or, you know, &etc.
z3c0 4y ago
I thought I must be going crazy until I saw your comment. This sounds like a bad research practice that probably shouldn't be reproduced to begin with.
- itsaquicknote 4y ago
  Research into systemically important infrastructure cannot be damned because that infrastructure isn't public. It's a cheap moralizing argument to say "pfff, this was predictable". Maybe so, but there isn't an alternative. Much like research on Twitter. Once these companies start to drift into providing what become broadscale social utilities and public services it doesn't matter that they're private. There are(/should be) obligations that come with that. You can't handwave and say go do your research on some micro-niche open source project that's way behind the SOTA and has nowhere near the same reach. That's not what "best practice" means here.

drexlspivey 4y ago

> we didn’t realize how important code-davinci-002 was to researchers, so we are keeping it going in our researcher access program: https://openai.com/form/researcher-access-program https://openai.com/form/researcher-access-program > we are also providing researcher access to the base GPT-4 model! https://twitter.com/sama/status/1638576434485825536 https://twitter.com/sama/status/1638576434485825536

varispeed 4y ago

The question is why would you start writing a paper based on a model you don't have control over? Surely the right way to do it, is to get your university to fund a creation of such a model first and then do research? Seems like researchers didn't think this through.

Xelynega 4y ago
Do research on the state of the art technology, or a homegrown copy of it that likely doesn't exhibit the same features. It seems like they thought it through to me.

deleted 4y ago

[deleted]