11 ms·

This seems an important article, if for no other reason than it brings the betrayal of its foundational claim still brazenly present in OpenAI's name from the o

This seems an important article, if for no other reason than it brings the betrayal of its foundational claim still brazenly present in OpenAI's name from the obscurity of HN comments going back years into the public light and the mainstream. They've achieved marvellous things, OpenAI, but the pivot and long-standing refusal to deal with it honestly leaves an unpleasant taste, and doesn't bode well for the future, especially considering the enormous ethical implications of advantage in the field they are leading.

asah 4y ago
Yay for competition.
- whywhywhywhy 4y ago
  Stability straight up proved that OpenAIs ideas around the importance of locking the tech up and guard railing it is all a big waste of time. The world didn’t end when anyone could run Dall-E 2 level image gen on gamer hardware and without guardrails. Instead we got to integrate that power into tools like Blender, Photoshop, Krita etc for free. First company to democratize ChatGPT tech in the same way will own this space and OpenAIs offering will once again become irrelevant overnight.
  - jackmott42 4y ago
    I generate Dall-E 2 level images on gamer hardware without guardrails.
    - devin 4y ago
      That’s exactly what OP said.
  - ben_w 4y ago
    That's like saying: > Ford straight up proved that Béla Barényi (of Mercedes Benz) ideas around crumple zones is all a big waste of time. The world didn't end with the 1938 Ford Prefect[0]. The world won't end overnight with an open fork of ChatGPT. But it will mean the signal-to-noise ratio rapidly shifts, that spammers and scammers will be much more effective, and that even minor special interest groups (or individuals) get the ability to cheaply fake a diverse crowd of people to support any cause at a slightly higher standard of discourse than the current waterline for random internet comments. [0] I don't know for certain it didn't have a crumple zone, but given when the patent was granted to Mercedes Benz…
    - whywhywhywhy 4y ago
      That car analogy doesn't work because Mercedes Benz didn't actively work against other cars from existing. https://www.vice.com/en/article/dy7nby/researchers-think-ai-language-models-will-help-spread-propaganda https://www.vice.com/en/article/dy7nby/researchers-think-ai-...
      - ben_w 4y ago
        First: Crumple zones have since become mandatory, which pattern matches what would happen if OpenAI gets what they ask for in that link. Second: I'm just as concerned about automated generation of propaganda as they seem to be. Given what LLMs are currently capable of doing, a free cyber-Goebbels for every hate group is the default: the AI itself only cares about predicting the next token, not the impact of having done so. Edit: Also, the headline of the Vice story you linked to is misleading given the source document that the body linked to. 1. Of the 6 researchers listed as authors of that report, only 2 are from OpenAI 2. Reduced exports of chips from the USA are discussed only briefly within that report, as part of a broader comparison with all the other possible ways to mitigate the various risks 3. Limited chip exports does nothing to prevent domestic propaganda and research https://cdn.openai.com/papers/forecasting-misuse.pdf https://cdn.openai.com/papers/forecasting-misuse.pdf
andrepd 4y ago
Seriously, is there anyone naive enough to have swallowed any of their trips? Corporations don't have ethical compasses, much less corporations founded by VCs aka professional salesmen.
- atlasunshrugged 4y ago
  Isn't that the point that it wasn't founded as a corporation, nor by Sam Altman? https://en.wikipedia.org/wiki/OpenAI#:~:text=The%20organization%20was%20founded%20in,collectively%20pledged%20US%241%20billion https://en.wikipedia.org/wiki/OpenAI#:~:text=The%20organizat....
  - ohgodplsno 4y ago
    [flagged]
    - uoaei 4y ago
      > There was no other end state for it. I agree, but I am curious and have a question for the less cynical: how could it have gone another way? How does one run a company that requires surviving in a harsh market that avoids these seeming inevitabilities? And in the case of the people running it, what made you think it would be different from their other ventures?
      - rhtgrg 4y ago
        ?
        uoaei 4y ago
        Any cursory glance at their funding streams makes it pretty obvious how the underlying business incentives express themselves. This isn't an "all companies are bad" argument, it's a "duh, that's how business works" argument. If your founding team is full of VCs and they decide they need to pay top wages using other VCs' money, well, maybe you can take it from there.
rippercushions 4y ago
To quote Spaceballs, they're not doing it for money, they're doing it for a shitload of money.
- 93po 4y ago
  OpenAI, if successful, will likely become the most valuable company in the history of the planet, both past and future.
  - ratorx 4y ago
    Really? I see a lot of competition in the space. I think they’d have to become significantly more successful than their competitors (some of which are massive AI powerhouses themselves) to achieve the market dominance necessary.
    - TeMPOraL 4y ago
      I think what GP is saying is that success of OpenAI means making a lot of profit, and then triggering the AI apocalypse - which is how they become most valuable company in history both past and future.
      - novaRom 4y ago
        Can you elaborate, what is "the AI apocalypse"? Is it just a symbolic metaphor or is there any scientific research behind this words? For me it's rather more unpredictable toxic environment we observe in the world currently, dominated by purely human-made destructive decisions, often based on purely animal instincts.
        flangola7 4y ago
        Without AI control loss: Permanent dictatorship by whoever controls AI. With AI control loss: AI converts the atoms of the observable universe to maximally achieve whatever arbitrary goal it thinks it was given. These are natural conclusions that can be drawn from the implied abilities of a superintelligent entity.
adamsmith143 4y ago
>This seems an important article, if for no other reason than it brings the betrayal of its foundational claim still brazenly present in OpenAI's name from the obscurity of HN comments going back years into the public light and the mainstream. The thing is, it's probably a good thing. The "Open" part of OpenAI was always an almost suicidal bad mistake. The point of starting the company was to try and reduce p(doom) of creating an AGI and it's almost certain that the more people have access to powerful and potentially dangerous tech that p(doom) increases. I think OpenAI is one of the most dangerous organizations on the planet and being more closed reduces that danger slightly.
- fragsworth 4y ago
  I'm curious what you think makes them dangerous?
  - therealdrag0 4y ago
    For long form, I’d suggest cold-takes blog who is very systematic thinker and has been focusing on agi risk recently. https://www.cold-takes.com https://www.cold-takes.com
    - fragsworth 4y ago
      I see a lot of "we don't know how it works therefore it could destroy all of us" but that sounds really handwavy to me. I want to see some concrete examples of how it's dangerous.
      - pixl97 4y ago
        Are people dangerous? Yes or no question. Do we have shitloads of regulations on what people can or cannot do? Yes or no question.
        fragsworth 4y ago
        Sometimes yes, and sometimes yes. I can be convinced, I just want to see the arguments.
IAmNotACellist 4y ago
It is nice to see normies noticing and caring, but the article leaves out some details that obscure comments still stubbornly bring up: like Musk founded it as a 501(c)(3) and put Altman in charge, and only once he had to leave with conflicts of interest Altman founded "OpenAI LP," the for-profit workaround so they didn't have to obey those pesky charity rules. That's when they stopped releasing models and weights, and started making their transparent claims that "the most ethical way to give people access to charge them fucktons of money and rip the API away when we feel like it."
- camillomiller 4y ago
  People keep forgetting we're talking about Sam Altman, someone who believed that scanning the world's population retinas in exchange for some crappy digital coin was a great idea and not a creepy spinoff of a shoddy sci-fi novel.
  - Loughla 4y ago
    Oh wow. I forgot that was even a thing. Is that project dead? I assume it is, but would love to be surprised that it still exists, just for the novelty of the entire thing.
    - zer0tonin 4y ago
      It's still around afaik
    - oneng 4y ago
      It's not merely still around, they're still actively hiring for it on LinkedIn.
    - shrimpx 4y ago
      Here it is: https://worldcoin.org/ https://worldcoin.org/
      - donnowhy 4y ago
wolpoli 4y ago
I feel betrayed whenever these organizations initially toot their own horns on how they are open or they are a charity, but then later on, change their goals or corporate structure with a vague press release. They then continue to keep the same name, allowing them to ride on their goodwill.
- dgs_sgd 4y ago
  And it's easy to believe that the betrayal was their intention all along as soon as they saw technological breakthroughs that could be monetized. It's hard to see why Sam Altman would be interested in non profits given his background in startups.
iKlsR 4y ago
Random aside, they recently bought https://ai.com https://ai.com which redirects to chat gpt atm but it wouldn't surprise me if they plan to drop the open eventually or move their paid stuff to it as they do have open source stuff that merits the name.
choppaface 4y ago
Also very notable that this post on HN now has nearly 1500 upvotes & 500 comments versus the OpenAI pricing launch article that has only 970 upvotes & 500 comments. HN Flameware posts are usually close to 1 upvote per comment, or more comments than upvotes, even. And yet this post has been pushed off the front page, and the pricing post remains. YC working the article ranking again!

asah 4y ago

Yay for competition.

whywhywhywhy 4y ago
Stability straight up proved that OpenAIs ideas around the importance of locking the tech up and guard railing it is all a big waste of time. The world didn’t end when anyone could run Dall-E 2 level image gen on gamer hardware and without guardrails. Instead we got to integrate that power into tools like Blender, Photoshop, Krita etc for free. First company to democratize ChatGPT tech in the same way will own this space and OpenAIs offering will once again become irrelevant overnight.
- jackmott42 4y ago
  I generate Dall-E 2 level images on gamer hardware without guardrails.
  - devin 4y ago
    That’s exactly what OP said.
- ben_w 4y ago
  That's like saying: > Ford straight up proved that Béla Barényi (of Mercedes Benz) ideas around crumple zones is all a big waste of time. The world didn't end with the 1938 Ford Prefect[0]. The world won't end overnight with an open fork of ChatGPT. But it will mean the signal-to-noise ratio rapidly shifts, that spammers and scammers will be much more effective, and that even minor special interest groups (or individuals) get the ability to cheaply fake a diverse crowd of people to support any cause at a slightly higher standard of discourse than the current waterline for random internet comments. [0] I don't know for certain it didn't have a crumple zone, but given when the patent was granted to Mercedes Benz…
  - whywhywhywhy 4y ago
    That car analogy doesn't work because Mercedes Benz didn't actively work against other cars from existing. https://www.vice.com/en/article/dy7nby/researchers-think-ai-language-models-will-help-spread-propaganda https://www.vice.com/en/article/dy7nby/researchers-think-ai-...
    - ben_w 4y ago
      First: Crumple zones have since become mandatory, which pattern matches what would happen if OpenAI gets what they ask for in that link. Second: I'm just as concerned about automated generation of propaganda as they seem to be. Given what LLMs are currently capable of doing, a free cyber-Goebbels for every hate group is the default: the AI itself only cares about predicting the next token, not the impact of having done so. Edit: Also, the headline of the Vice story you linked to is misleading given the source document that the body linked to. 1. Of the 6 researchers listed as authors of that report, only 2 are from OpenAI 2. Reduced exports of chips from the USA are discussed only briefly within that report, as part of a broader comparison with all the other possible ways to mitigate the various risks 3. Limited chip exports does nothing to prevent domestic propaganda and research https://cdn.openai.com/papers/forecasting-misuse.pdf https://cdn.openai.com/papers/forecasting-misuse.pdf

andrepd 4y ago

Seriously, is there anyone naive enough to have swallowed any of their trips? Corporations don't have ethical compasses, much less corporations founded by VCs aka professional salesmen.

atlasunshrugged 4y ago
Isn't that the point that it wasn't founded as a corporation, nor by Sam Altman? https://en.wikipedia.org/wiki/OpenAI#:~:text=The%20organization%20was%20founded%20in,collectively%20pledged%20US%241%20billion https://en.wikipedia.org/wiki/OpenAI#:~:text=The%20organizat....
- ohgodplsno 4y ago
  [flagged]
  - uoaei 4y ago
    > There was no other end state for it. I agree, but I am curious and have a question for the less cynical: how could it have gone another way? How does one run a company that requires surviving in a harsh market that avoids these seeming inevitabilities? And in the case of the people running it, what made you think it would be different from their other ventures?
    - rhtgrg 4y ago
      ?
      - uoaei 4y ago
        Any cursory glance at their funding streams makes it pretty obvious how the underlying business incentives express themselves. This isn't an "all companies are bad" argument, it's a "duh, that's how business works" argument. If your founding team is full of VCs and they decide they need to pay top wages using other VCs' money, well, maybe you can take it from there.

rippercushions 4y ago

To quote Spaceballs, they're not doing it for money, they're doing it for a shitload of money.

93po 4y ago
OpenAI, if successful, will likely become the most valuable company in the history of the planet, both past and future.
- ratorx 4y ago
  Really? I see a lot of competition in the space. I think they’d have to become significantly more successful than their competitors (some of which are massive AI powerhouses themselves) to achieve the market dominance necessary.
  - TeMPOraL 4y ago
    I think what GP is saying is that success of OpenAI means making a lot of profit, and then triggering the AI apocalypse - which is how they become most valuable company in history both past and future.
    - novaRom 4y ago
      Can you elaborate, what is "the AI apocalypse"? Is it just a symbolic metaphor or is there any scientific research behind this words? For me it's rather more unpredictable toxic environment we observe in the world currently, dominated by purely human-made destructive decisions, often based on purely animal instincts.
      - flangola7 4y ago
        Without AI control loss: Permanent dictatorship by whoever controls AI. With AI control loss: AI converts the atoms of the observable universe to maximally achieve whatever arbitrary goal it thinks it was given. These are natural conclusions that can be drawn from the implied abilities of a superintelligent entity.

adamsmith143 4y ago

>This seems an important article, if for no other reason than it brings the betrayal of its foundational claim still brazenly present in OpenAI's name from the obscurity of HN comments going back years into the public light and the mainstream. The thing is, it's probably a good thing. The "Open" part of OpenAI was always an almost suicidal bad mistake. The point of starting the company was to try and reduce p(doom) of creating an AGI and it's almost certain that the more people have access to powerful and potentially dangerous tech that p(doom) increases. I think OpenAI is one of the most dangerous organizations on the planet and being more closed reduces that danger slightly.

fragsworth 4y ago
I'm curious what you think makes them dangerous?
- therealdrag0 4y ago
  For long form, I’d suggest cold-takes blog who is very systematic thinker and has been focusing on agi risk recently. https://www.cold-takes.com https://www.cold-takes.com
  - fragsworth 4y ago
    I see a lot of "we don't know how it works therefore it could destroy all of us" but that sounds really handwavy to me. I want to see some concrete examples of how it's dangerous.
    - pixl97 4y ago
      Are people dangerous? Yes or no question. Do we have shitloads of regulations on what people can or cannot do? Yes or no question.
      - fragsworth 4y ago
        Sometimes yes, and sometimes yes. I can be convinced, I just want to see the arguments.

IAmNotACellist 4y ago

It is nice to see normies noticing and caring, but the article leaves out some details that obscure comments still stubbornly bring up: like Musk founded it as a 501(c)(3) and put Altman in charge, and only once he had to leave with conflicts of interest Altman founded "OpenAI LP," the for-profit workaround so they didn't have to obey those pesky charity rules. That's when they stopped releasing models and weights, and started making their transparent claims that "the most ethical way to give people access to charge them fucktons of money and rip the API away when we feel like it."

camillomiller 4y ago
People keep forgetting we're talking about Sam Altman, someone who believed that scanning the world's population retinas in exchange for some crappy digital coin was a great idea and not a creepy spinoff of a shoddy sci-fi novel.
- Loughla 4y ago
  Oh wow. I forgot that was even a thing. Is that project dead? I assume it is, but would love to be surprised that it still exists, just for the novelty of the entire thing.
  - zer0tonin 4y ago
    It's still around afaik
  - oneng 4y ago
    It's not merely still around, they're still actively hiring for it on LinkedIn.
  - shrimpx 4y ago
    Here it is: https://worldcoin.org/ https://worldcoin.org/
    - donnowhy 4y ago

wolpoli 4y ago

I feel betrayed whenever these organizations initially toot their own horns on how they are open or they are a charity, but then later on, change their goals or corporate structure with a vague press release. They then continue to keep the same name, allowing them to ride on their goodwill.

dgs_sgd 4y ago
And it's easy to believe that the betrayal was their intention all along as soon as they saw technological breakthroughs that could be monetized. It's hard to see why Sam Altman would be interested in non profits given his background in startups.

iKlsR 4y ago

Random aside, they recently bought https://ai.com https://ai.com which redirects to chat gpt atm but it wouldn't surprise me if they plan to drop the open eventually or move their paid stuff to it as they do have open source stuff that merits the name.

choppaface 4y ago

Also very notable that this post on HN now has nearly 1500 upvotes & 500 comments versus the OpenAI pricing launch article that has only 970 upvotes & 500 comments. HN Flameware posts are usually close to 1 upvote per comment, or more comments than upvotes, even. And yet this post has been pushed off the front page, and the pricing post remains. YC working the article ranking again!