5 ms·

> We’re introducing our video generation technology now to give society time to explore its possibilities and co-develop norms and safeguards that ensure it’s u

> We’re introducing our video generation technology now to give society time to explore its possibilities and co-develop norms and safeguards that ensure it’s used responsibly as the field advances. That's an interesting way of saying "we're probably gonna miss some stuff in our safety tools, so hopefully society picks up the slack for us". :)

FrustratedMonky 2y ago
"to give society time to explore its possibilities and co-develop norms and safeguards" Or, "this safety stuff is harder than we thought, we're just going to call 'tag you're it' on society" Or, -Oppenheimer : speaking "man, this nuclear safety stuff is hard, I'm just going to put it all out there and let society explore developing norms and safeguards". -Society : Bombs Japan -Oppenheimer : "No, not like that, oops".
- deleted 2y ago
  [deleted]
- Arnt 2y ago
  Aren't you kind of saying that you don't have any answers so therefore OpenAI should have provided the answers?
- usrnm 2y ago
  Oppenheimer was making a bomb from day 1, he knew exactly what he was doing and how it would be used. There aren't so many different use cases for a bomb, after all. It was a nice movie, but it does not absolve him
- xvector 2y ago
  Eh, society did a pretty good job overall. The bomb was the end of conventional warfare between nuclear nations. MAD has created an era of peace unlike anything our species has ever seen before.
  - rurp 2y ago
    Well it works great, until is doesn't. We're perpetually a few bad decisions from a few possibly deranged actors away from obliterating all of those gains and then some.
    - xvector 2y ago
      Right, and in the meantime nuclear-armed countries mostly get to avoid the horrible, endless churn of death and war and teenagers being sent off to the meat grinder to push some border here or some border there. We have eliminated warfare between nuclear countries, conflicts have been reduced to nuclear/non-nuclear or proxy warfare, and that's a very solid reduction in suffering.
jsheard 2y ago
Flashbacks to when they were cagey about releasing the GPT models because they could so easily be used for spam, and then just pretended not to see all the spam their model was making when they did release it. If you happen to notice a Twitter spam bot claiming to be "an AI language model created by OpenAI", know that we have conducted an investigation and concluded that no you didn't. Mission accomplished!
- echelon 2y ago
  > not to see all the spam their model was making when they did release it. All replaced by open source LLMs at this point. Most AI video will be produced by Hunyuan [1], LTX [2], and Mochi [3] in short order. These are the Flux / Stable Diffusion models for generative video. These can all be fine tuned to produce incredible results, and work with the Comfy ecosystem for wildly creative and controllable workflows. I don't think it'll be possible for a closed source tool to compete with the open image/video ecosystem. Dall-E certainly didn't stay competitive for long. It's a totally different game. [1] https://github.com/Tencent/HunyuanVideo https://github.com/Tencent/HunyuanVideo [2] https://huggingface.co/Lightricks/LTX-Video https://huggingface.co/Lightricks/LTX-Video [3] https://github.com/genmoai/mochi https://github.com/genmoai/mochi
  - jsheard 2y ago
    > I don't think it'll be possible for a closed source tool to compete with the open image/video ecosystem. And I don't think the current status quo of open source models being entirely subsidised by startups and corporations is sustainable, they're all hemorrhaging money and their investors will only have so much patience before they expect returns. Enjoy it while it lasts.
    - echelon 2y ago
      It's game theory. If you don't have market share for your closed model, you release it as open source and let a community build upon it. Mochi is better positioned to build tools on top of their community model. They're already thinking about control. Weights are commodity. Products have value.
nostromo 2y ago
The irony is that users want more freedom and fewer safeguards. But these companies are rightfully worried about regulators and legislatures, often led by a pearl-clutching journalists, so we can't have nice things.
- throawayonthe 2y ago
  [dead]
- DFHippie 2y ago
  Recent events (many events in many places) show "users" don't think too hard before acting. And sometimes they act with inadequate or inaccurate information. If we want better outcomes, it behooves us to hire people to do the thinking that ordinary users see no point in doing for themselves. We call the people doing the hard thinking scientists, regulators, and journalists. The regulators, when empowered to do so by the government, can stop things from happening. The scientists and journalists can just issue warnings. Giving people what they want when they want it doesn't always lead to happy outcomes. The people themselves, through their representatives, have created the institutions that sometimes put a brake on their worst impulses.
nicbou 2y ago
The onus will be on the rest of society to defend itself from all the grift that will result from this.
- pesus 2y ago
  If the worst we ultimately get from this kind of tech is grifting, I will consider that a very positive outcome.
miohtama 2y ago
Users, not tools, should be judged. It is unlikely anyone is going to perform act of terrorism with this, or any kind of deep fakes that buy Easter European elections. The worst outcome is likely teens having a laugh.
- observationist 2y ago
  Funny how all the negative uses to which something like this might be put are regulated or criminalized already - if you try to scam someone, commit libel or defamation, attempt widespread fraud, or any of a million nefarious uses, you'll get fined, sued, or go to jail. Would you want Microsoft to claim they're responsible for the "safety" of what you write with Word? For the legality of the numbers you're punching into an Excel spreadsheet? Would you want Verizon keeping tabs on every word you say, to make sure it's in line with their corporate ethos? This idea that AI is somehow special, that they absolutely must monitor and censor and curtail usage, that they claim total responsibility for the behavior of their users - Anthropic and OpenAI don't seem to realize that they're the bad guys. If you build tools of totalitarian dystopian tyranny, dystopian tyrants will take those tools from you and use them. Or worse yet, force your compliance and you'll become nothing more than the big stick used to keep people cowed. We have laws and norms and culture about what's ok and what's not ok to write, produce, and publish. We don't need corporate morality police, thanks. Censorship of tools is ethically wrong. If someone wants to publish things that are horrific or illegal, let that person be responsible for their own actions. There is absolutely no reason for AI companies to be involved.
  - 8note 2y ago
    that works for locally hosted models, but if its as a service, openai is publishing those verboten works to you, the person who requested it. even if it is a local model, if you trained a model to spew nazi propaganda, youre still publishing nazi propaganda to the people who then go use it to make propaganda. its just very summarized propaganda
    - gus_massa 2y ago
      Does this apply to the spell checker in Office 365 or Google Docs?
sleepybrett 2y ago
'when civilization collapses because all photo, audio and video evidence is 100% suspect, i mean, how could you blame us'
jstummbillig 2y ago
Do we not want new stuff? If the answer is "Sure, but only if whoever invents the stuff does all the work and finds all rough edges" then the answer is actually just "No, thanks".
- jedberg 2y ago
  Oh, I have no problem with them doing it this way. I just thought it was a funny way to do it.
- kevingadd 2y ago
  It's a little disingenuous to jump to "we don't want new stuff" when people voice criticism of deepfake generators or AI models trained on stolen content
123yawaworht456 2y ago
text, image, video, and audio editing tools have no 'safety' and 'alignment' whatsoever, and skilled humans are far more capable of creating 'unsafe' and 'unethical' media than generative AI will ever be. somehow, the society had survived just fine. the notion that generative AI tools should be 'safe' and 'aligned' is as absurd as the notion that tools like Notepad, Photoshop, Premiere and Audacity should exist only in the cloud, monitored by kommissars to ensure that proles aren't doing something 'unsafe' with them.
pyrale 2y ago
"We're releasing this like rats on a remote island, in hopes of seeing how the ecosystem is going to respond".
raincole 2y ago
The problem isn't whether we should regulate AI. It's whether it's even possible to regulate them without causing significant turmoil and damage to the society. It's not hyperbole. Hunyuan was released before Sora. So regulating Sora does absolutely nothing unless you can regulate Hunyuan, which is 1) open source and 2) made by a China company. How do we expect the US govt to regulate that? Threatening sanction China unless they stop doing AI research???
- ssl-3 2y ago
  Easy-peasy. Just require all software to be cryptographically signed, with a trusted chain that leads to a government-vetted author, and make that author responsible for the wrongdoings of that software's users. We're most of the way there with "our" locked-down, walled-garden pocket supercomputers. Just extend that breadth and bring it to the rest of computing using the force of law. --- Can I hear someone saying something like "That will never work!"? Perhaps we should meditate upon that before we leap into any new age of regulation.
  - kaszanka 2y ago
    This is well on its way thanks to Microsoft's aggressive push to put a TPM in every Windows 11 PC.
    - ssl-3 2y ago
      That's exactly the kind of logical conclusion I had hoped for someone here to reach in this bizarre sea of emotional pleas. After over two decades of careful preparation, we're the stroke of a legislative pen away from having all of the software on our computers regulated by our friends in the government. It's not even a slippery slope argument. In order to be effective, "We must regulate AI!" means the same thing as "We must regulate computer software!" The two things are so identical that they're not even so different as two sides of the same coin are. (Be careful what you wish for; you might just get it.)
Nition 2y ago
"Climate Change is likely to mean more fires in the future, so we've lit a small fire at everyone's house to give society time to co-develop norms and safeguards."
soheil 2y ago
Specially since they were originally supposed to be a non-profit focused on AI safety and Sam Altman single-handedly pivoting to a for-profit after taking all the donations and partnering with probably the single most evil corporation that has ever existed, Microsoft.
- lukevp 2y ago
  Microsoft is more evil than Enron? Than the company that faked blood tests? This is some pretty extreme hyperbole. I’d pick Google over Microsoft for one.
golemiprague 2y ago
[dead]

FrustratedMonky 2y ago

"to give society time to explore its possibilities and co-develop norms and safeguards" Or, "this safety stuff is harder than we thought, we're just going to call 'tag you're it' on society" Or, -Oppenheimer : speaking "man, this nuclear safety stuff is hard, I'm just going to put it all out there and let society explore developing norms and safeguards". -Society : Bombs Japan -Oppenheimer : "No, not like that, oops".

deleted 2y ago
[deleted]
Arnt 2y ago
Aren't you kind of saying that you don't have any answers so therefore OpenAI should have provided the answers?
usrnm 2y ago
Oppenheimer was making a bomb from day 1, he knew exactly what he was doing and how it would be used. There aren't so many different use cases for a bomb, after all. It was a nice movie, but it does not absolve him
xvector 2y ago
Eh, society did a pretty good job overall. The bomb was the end of conventional warfare between nuclear nations. MAD has created an era of peace unlike anything our species has ever seen before.
- rurp 2y ago
  Well it works great, until is doesn't. We're perpetually a few bad decisions from a few possibly deranged actors away from obliterating all of those gains and then some.
  - xvector 2y ago
    Right, and in the meantime nuclear-armed countries mostly get to avoid the horrible, endless churn of death and war and teenagers being sent off to the meat grinder to push some border here or some border there. We have eliminated warfare between nuclear countries, conflicts have been reduced to nuclear/non-nuclear or proxy warfare, and that's a very solid reduction in suffering.

jsheard 2y ago

Flashbacks to when they were cagey about releasing the GPT models because they could so easily be used for spam, and then just pretended not to see all the spam their model was making when they did release it. If you happen to notice a Twitter spam bot claiming to be "an AI language model created by OpenAI", know that we have conducted an investigation and concluded that no you didn't. Mission accomplished!

echelon 2y ago
> not to see all the spam their model was making when they did release it. All replaced by open source LLMs at this point. Most AI video will be produced by Hunyuan [1], LTX [2], and Mochi [3] in short order. These are the Flux / Stable Diffusion models for generative video. These can all be fine tuned to produce incredible results, and work with the Comfy ecosystem for wildly creative and controllable workflows. I don't think it'll be possible for a closed source tool to compete with the open image/video ecosystem. Dall-E certainly didn't stay competitive for long. It's a totally different game. [1] https://github.com/Tencent/HunyuanVideo https://github.com/Tencent/HunyuanVideo [2] https://huggingface.co/Lightricks/LTX-Video https://huggingface.co/Lightricks/LTX-Video [3] https://github.com/genmoai/mochi https://github.com/genmoai/mochi
- jsheard 2y ago
  > I don't think it'll be possible for a closed source tool to compete with the open image/video ecosystem. And I don't think the current status quo of open source models being entirely subsidised by startups and corporations is sustainable, they're all hemorrhaging money and their investors will only have so much patience before they expect returns. Enjoy it while it lasts.
  - echelon 2y ago
    It's game theory. If you don't have market share for your closed model, you release it as open source and let a community build upon it. Mochi is better positioned to build tools on top of their community model. They're already thinking about control. Weights are commodity. Products have value.

nostromo 2y ago

The irony is that users want more freedom and fewer safeguards. But these companies are rightfully worried about regulators and legislatures, often led by a pearl-clutching journalists, so we can't have nice things.

throawayonthe 2y ago
[dead]
DFHippie 2y ago
Recent events (many events in many places) show "users" don't think too hard before acting. And sometimes they act with inadequate or inaccurate information. If we want better outcomes, it behooves us to hire people to do the thinking that ordinary users see no point in doing for themselves. We call the people doing the hard thinking scientists, regulators, and journalists. The regulators, when empowered to do so by the government, can stop things from happening. The scientists and journalists can just issue warnings. Giving people what they want when they want it doesn't always lead to happy outcomes. The people themselves, through their representatives, have created the institutions that sometimes put a brake on their worst impulses.

nicbou 2y ago

The onus will be on the rest of society to defend itself from all the grift that will result from this.

pesus 2y ago
If the worst we ultimately get from this kind of tech is grifting, I will consider that a very positive outcome.

miohtama 2y ago

Users, not tools, should be judged. It is unlikely anyone is going to perform act of terrorism with this, or any kind of deep fakes that buy Easter European elections. The worst outcome is likely teens having a laugh.

observationist 2y ago
Funny how all the negative uses to which something like this might be put are regulated or criminalized already - if you try to scam someone, commit libel or defamation, attempt widespread fraud, or any of a million nefarious uses, you'll get fined, sued, or go to jail. Would you want Microsoft to claim they're responsible for the "safety" of what you write with Word? For the legality of the numbers you're punching into an Excel spreadsheet? Would you want Verizon keeping tabs on every word you say, to make sure it's in line with their corporate ethos? This idea that AI is somehow special, that they absolutely must monitor and censor and curtail usage, that they claim total responsibility for the behavior of their users - Anthropic and OpenAI don't seem to realize that they're the bad guys. If you build tools of totalitarian dystopian tyranny, dystopian tyrants will take those tools from you and use them. Or worse yet, force your compliance and you'll become nothing more than the big stick used to keep people cowed. We have laws and norms and culture about what's ok and what's not ok to write, produce, and publish. We don't need corporate morality police, thanks. Censorship of tools is ethically wrong. If someone wants to publish things that are horrific or illegal, let that person be responsible for their own actions. There is absolutely no reason for AI companies to be involved.
- 8note 2y ago
  that works for locally hosted models, but if its as a service, openai is publishing those verboten works to you, the person who requested it. even if it is a local model, if you trained a model to spew nazi propaganda, youre still publishing nazi propaganda to the people who then go use it to make propaganda. its just very summarized propaganda
  - gus_massa 2y ago
    Does this apply to the spell checker in Office 365 or Google Docs?

sleepybrett 2y ago

'when civilization collapses because all photo, audio and video evidence is 100% suspect, i mean, how could you blame us'

jstummbillig 2y ago

Do we not want new stuff? If the answer is "Sure, but only if whoever invents the stuff does all the work and finds all rough edges" then the answer is actually just "No, thanks".

jedberg 2y ago
Oh, I have no problem with them doing it this way. I just thought it was a funny way to do it.
kevingadd 2y ago
It's a little disingenuous to jump to "we don't want new stuff" when people voice criticism of deepfake generators or AI models trained on stolen content

123yawaworht456 2y ago

text, image, video, and audio editing tools have no 'safety' and 'alignment' whatsoever, and skilled humans are far more capable of creating 'unsafe' and 'unethical' media than generative AI will ever be. somehow, the society had survived just fine. the notion that generative AI tools should be 'safe' and 'aligned' is as absurd as the notion that tools like Notepad, Photoshop, Premiere and Audacity should exist only in the cloud, monitored by kommissars to ensure that proles aren't doing something 'unsafe' with them.

pyrale 2y ago

"We're releasing this like rats on a remote island, in hopes of seeing how the ecosystem is going to respond".

raincole 2y ago

The problem isn't whether we should regulate AI. It's whether it's even possible to regulate them without causing significant turmoil and damage to the society. It's not hyperbole. Hunyuan was released before Sora. So regulating Sora does absolutely nothing unless you can regulate Hunyuan, which is 1) open source and 2) made by a China company. How do we expect the US govt to regulate that? Threatening sanction China unless they stop doing AI research???

ssl-3 2y ago
Easy-peasy. Just require all software to be cryptographically signed, with a trusted chain that leads to a government-vetted author, and make that author responsible for the wrongdoings of that software's users. We're most of the way there with "our" locked-down, walled-garden pocket supercomputers. Just extend that breadth and bring it to the rest of computing using the force of law. --- Can I hear someone saying something like "That will never work!"? Perhaps we should meditate upon that before we leap into any new age of regulation.
- kaszanka 2y ago
  This is well on its way thanks to Microsoft's aggressive push to put a TPM in every Windows 11 PC.
  - ssl-3 2y ago
    That's exactly the kind of logical conclusion I had hoped for someone here to reach in this bizarre sea of emotional pleas. After over two decades of careful preparation, we're the stroke of a legislative pen away from having all of the software on our computers regulated by our friends in the government. It's not even a slippery slope argument. In order to be effective, "We must regulate AI!" means the same thing as "We must regulate computer software!" The two things are so identical that they're not even so different as two sides of the same coin are. (Be careful what you wish for; you might just get it.)

Nition 2y ago

"Climate Change is likely to mean more fires in the future, so we've lit a small fire at everyone's house to give society time to co-develop norms and safeguards."

soheil 2y ago

Specially since they were originally supposed to be a non-profit focused on AI safety and Sam Altman single-handedly pivoting to a for-profit after taking all the donations and partnering with probably the single most evil corporation that has ever existed, Microsoft.

lukevp 2y ago
Microsoft is more evil than Enron? Than the company that faked blood tests? This is some pretty extreme hyperbole. I’d pick Google over Microsoft for one.

golemiprague 2y ago

[dead]