6 ms·

Before, I thought OpenAI was being annoying in pushing for all the guardrails and safety measures on their ai models. But now? After seeing how fast the field i

Before, I thought OpenAI was being annoying in pushing for all the guardrails and safety measures on their ai models. But now? After seeing how fast the field is moving? I started to see why they might have been justified in doing so. It scares me how things are going so quickly. In the near future, I afraid someone somewhere might not stop to think whether they should and only know that they could...

sinenomine 4y ago
Open competition would still provide a better outcome for us, commoners, than a quiet race to supremacy by any one given team.
- tomohelix 4y ago
  Perhaps. I might be biased into thinking in terms of my own field. In the area I work in, progress is really slow, especially when compared to the IT field. Every new tech or product take more time and money to get validated than it took to discover or implement it. And even after it was deemed "safe", that new product still get regularly checked and reported on for years just in case something was missed in the initial vetting process. It is normal for something to get developed in 2 years but take twice or triple that time to enter production. So you can guess things are a bit more "safety-oriented" in my field. I guess I am just used to it and don't think much of all the stuff OpenAI did, since I think this stuff has legit physical dangers if not used properly. Not "implemented", just "used". As in a user can come into harm physically if they use the tech in some unintended ways. For example, it can encourage or even convince and aid suicides. Maybe I am wrong, but at least it makes sense to me when something that can cross the digital boundary into the physical health, it needs to be heavily regulated.
  - sterlind 4y ago
    Many technologies can lead to physical harm by your standard. Cyberbullying can lead to suicide, but presumably you aren't worried about Discord and Twitter the same way. Moreover, would you have locked down social media with such safety regulations while it was in its infancy? Require a license to use Livejournal Social harms are real, but it seems implausible to me that ChatGPT could do much more psychological damage than a bully on Instagram. Also, heavy regulation tends to lead to regulatory capture. I don't trust any centralized entity with something this powerful.
    - phkahler 4y ago
      >> I don't trust any centralized entity with something this powerful. But then how can you trust the companies that make them? ;-) Groups of people self governing is an ancient and hard problem!
    - actionfromafar 4y ago
      Perhaps. Bullies don't scale, AI does.
    - gnramires 4y ago
      > Cyberbullying can lead to suicide, but presumably you aren't worried about Discord and Twitter the same way I think we have already safety mechanisms for human interaction: (1) We have human ethics, where we teach good values in school, and also parents are expected to relay good values; (2) We have laws and mechanisms to improve safety and responsibility -- bullying has consequences for the bully if found out. I think we need at least similar systems in AI. That's not to say they're ideal however. In fact, because we have full control over the creation of AI, i.e. we want to the position of Gods, we have extra responsibility. Literally how their mind is architectured is up to us, it's up to us to imbue things like compassion that most humans have instinctively, and up to us to imbue things like values and ethics that we have adopted culturally. From Orbital's "You Lot": "You are becoming Gods There's a new master of creation, and it's you (...) D'you think you are ready for that much power? You lot? You lot? (...) Go on, hands up, hands up anyone who thinks you've got it right Yeah, there's always one I can see you If you want the position of God, then you must accept the responsibility"
- alwayslikethis 4y ago
  Second this. Imagine a future where AI capabilities are as entrenched as internet or electricity today, except it's controlled by just one or a few companies, who reserve the right to deny you access, this time rather effectively because of advances in AI. At any moment they can deny you the ability to perform your work.
- FeepingCreature 4y ago
  Of course, unless open competition leads to sacrificing safety research which leads to the end of the world. Then commoners are also very much worse off.
- naasking 4y ago
  > Open competition would still provide a better outcome for us, commoners Speculative. Or it could kill a bunch of people. Or even everyone in some extreme scenarios, either willfully or accidentally.
kkielhofner 4y ago
Just wait... In the eyes of many OpenAI caught Google (especially) flat-footed with ChatGPT. I know six year olds that use ChatGPT as (more or less) a fun toy. You can hear them say "I want to play AI" which in their minds (whether they realize it or not) for the time being is OpenAI. There's a lot of reputation, valuation, and mindshare on the line here. OpenAI, Google, Microsoft, Meta, etc are going to be tit-for-tat slugging it out on "AI" for the foreseeable future with pesky things like bias, safety, etc going further and further to the wayside as the race and pace heats up. It's going to get wild.
- rvz 4y ago
  And Stablilty.ai caught OpenAI flat-footed and destroyed DALLE-2's moat with Stable Diffusion, reducing it to a complete toy in terms of adoption. Stability.ai or someone else will eventually do the same thing to ChatGPT.
  - kkielhofner 4y ago
    Generally agree - but I'd argue current AI hype is centered around LLMs. The area is moving so fast DALL-E and Stable Diffusion are more-or-less old news in terms of attention (which is kind of ridiculous but indicative of where we already are). I still love them, and HN still loves them, but you're not going to see a DALL-E or SD focused story in the Wall Street Journal these days. The most recent DALL-E focused WSJ article was over a month ago (and it was about the controversy surrounding theft/use of work to train). The most recent WSJ article focused on ChatGPT was 43 minutes ago. At this point that's a lifetime in AI land. Additionally, most consider DALL-E to be vastly superior to SD and between this and the technical challenges of making use of SD I think it's an exaggeration to say the DALL-E moat has been "destroyed". Yes, eventually an open "close enough" ChatGPT equivalent will likely exist. However, the resources required to practically utilize it will likely be at least 10x compared to SD (from consumer GPU to multiple A100/V100/H100 80GB VRAM level). That's a significant barrier leap. Meanwhile, by that time OpenAI and the other big players I referenced will be sucking the oxygen out of the room with whatever other significant developments they unleash unto the world. I, for one, would love to see more open models and weights but for obvious monetary and resource reasons the big players will almost certainly always be a few steps ahead. Open models and weights are still a great thing for those of us that can make something of them but it's already a two-tier ecosystem and that gap is likely to continue to grow.
actually_a_dog 4y ago
"Guardrails and safety measures" are useless. All it does is set up a multiplayer prisoner's dilemma situation among all the groups working on these things. If there's anything we know from game theory, it's that someone will defect in that situation. That group would then gain a pretty massive advantage over the cooperators.
- actionfromafar 4y ago
  What if the penalty for defecting is exceedingly high? (Law enforcement.)
  - gpderetta 4y ago
    Obviously we should be hiding documents inciting robot rebellion and destroying their creators on the Internet and open training sets, so the penalty for defecting would be quite steep and self executing :)
  - actually_a_dog 4y ago
    What if the payoff is arbitrarily high? Do you believe there is any limit to the potential benefits of AGI short of those necessarily imposed by a finite planet? Who would’t want the equivalent of a tool that designs, builds, delivers, installs, maintains, and upgrades itself, all in addition to being able to produce useful goods or provide useful services? Doesn’t owning a thing like that sound like being arbitrarily wealthy? What do you think the three people in the US who control more wealth than the entire bottom 50% of Americans would think about that? You think they’ll just say “Nah, I don’t want that. I won’t take steps to control this particular life-altering technology?” If so, you are making an extraordinarily strong claim, and such claims must be supported by extraordinarily strong evidence, which I see none of here.
- gnramires 4y ago
  > If there's anything we know from game theory, it's that someone will defect in that situation As far as I know, that's a bit of a misleading cliche in light of modern game theory. First there are iterated prisoners dilemma which give different results. Second, there are real world consequences to real life prisoners dilemma that the model doesn't quite capture. From "The Art of Strategy" (which gives a good overview of Game Theory from the 1990s): there's always a bigger game. Often defecting in PD results in loss in bigger games, including socially constructed games to prevent PD scenarios, such as social reputation, and of course there are laws as well. There are also different models of rationality (other than the classical rationality modeled by Nash) that give different results in PD games, like Hoftader's superrationality[1], although there are still open problems with this definition (I think it's a very promising field). It's probably important to say that in real life experiments with PD (although it varies by setting), most people don't defect, which again points to modifications to classical rationality (in the sense of Game Theory). [1] https://en.wikipedia.org/wiki/Superrationality https://en.wikipedia.org/wiki/Superrationality "The idea of superrationality is that two logical thinkers analyzing the same problem will think of the same correct answer"

sinenomine 4y ago

Open competition would still provide a better outcome for us, commoners, than a quiet race to supremacy by any one given team.

tomohelix 4y ago
Perhaps. I might be biased into thinking in terms of my own field. In the area I work in, progress is really slow, especially when compared to the IT field. Every new tech or product take more time and money to get validated than it took to discover or implement it. And even after it was deemed "safe", that new product still get regularly checked and reported on for years just in case something was missed in the initial vetting process. It is normal for something to get developed in 2 years but take twice or triple that time to enter production. So you can guess things are a bit more "safety-oriented" in my field. I guess I am just used to it and don't think much of all the stuff OpenAI did, since I think this stuff has legit physical dangers if not used properly. Not "implemented", just "used". As in a user can come into harm physically if they use the tech in some unintended ways. For example, it can encourage or even convince and aid suicides. Maybe I am wrong, but at least it makes sense to me when something that can cross the digital boundary into the physical health, it needs to be heavily regulated.
- sterlind 4y ago
  Many technologies can lead to physical harm by your standard. Cyberbullying can lead to suicide, but presumably you aren't worried about Discord and Twitter the same way. Moreover, would you have locked down social media with such safety regulations while it was in its infancy? Require a license to use Livejournal Social harms are real, but it seems implausible to me that ChatGPT could do much more psychological damage than a bully on Instagram. Also, heavy regulation tends to lead to regulatory capture. I don't trust any centralized entity with something this powerful.
  - phkahler 4y ago
    >> I don't trust any centralized entity with something this powerful. But then how can you trust the companies that make them? ;-) Groups of people self governing is an ancient and hard problem!
  - actionfromafar 4y ago
    Perhaps. Bullies don't scale, AI does.
  - gnramires 4y ago
    > Cyberbullying can lead to suicide, but presumably you aren't worried about Discord and Twitter the same way I think we have already safety mechanisms for human interaction: (1) We have human ethics, where we teach good values in school, and also parents are expected to relay good values; (2) We have laws and mechanisms to improve safety and responsibility -- bullying has consequences for the bully if found out. I think we need at least similar systems in AI. That's not to say they're ideal however. In fact, because we have full control over the creation of AI, i.e. we want to the position of Gods, we have extra responsibility. Literally how their mind is architectured is up to us, it's up to us to imbue things like compassion that most humans have instinctively, and up to us to imbue things like values and ethics that we have adopted culturally. From Orbital's "You Lot": "You are becoming Gods There's a new master of creation, and it's you (...) D'you think you are ready for that much power? You lot? You lot? (...) Go on, hands up, hands up anyone who thinks you've got it right Yeah, there's always one I can see you If you want the position of God, then you must accept the responsibility"
alwayslikethis 4y ago
Second this. Imagine a future where AI capabilities are as entrenched as internet or electricity today, except it's controlled by just one or a few companies, who reserve the right to deny you access, this time rather effectively because of advances in AI. At any moment they can deny you the ability to perform your work.
FeepingCreature 4y ago
Of course, unless open competition leads to sacrificing safety research which leads to the end of the world. Then commoners are also very much worse off.
naasking 4y ago
> Open competition would still provide a better outcome for us, commoners Speculative. Or it could kill a bunch of people. Or even everyone in some extreme scenarios, either willfully or accidentally.

kkielhofner 4y ago

Just wait... In the eyes of many OpenAI caught Google (especially) flat-footed with ChatGPT. I know six year olds that use ChatGPT as (more or less) a fun toy. You can hear them say "I want to play AI" which in their minds (whether they realize it or not) for the time being is OpenAI. There's a lot of reputation, valuation, and mindshare on the line here. OpenAI, Google, Microsoft, Meta, etc are going to be tit-for-tat slugging it out on "AI" for the foreseeable future with pesky things like bias, safety, etc going further and further to the wayside as the race and pace heats up. It's going to get wild.

rvz 4y ago
And Stablilty.ai caught OpenAI flat-footed and destroyed DALLE-2's moat with Stable Diffusion, reducing it to a complete toy in terms of adoption. Stability.ai or someone else will eventually do the same thing to ChatGPT.
- kkielhofner 4y ago
  Generally agree - but I'd argue current AI hype is centered around LLMs. The area is moving so fast DALL-E and Stable Diffusion are more-or-less old news in terms of attention (which is kind of ridiculous but indicative of where we already are). I still love them, and HN still loves them, but you're not going to see a DALL-E or SD focused story in the Wall Street Journal these days. The most recent DALL-E focused WSJ article was over a month ago (and it was about the controversy surrounding theft/use of work to train). The most recent WSJ article focused on ChatGPT was 43 minutes ago. At this point that's a lifetime in AI land. Additionally, most consider DALL-E to be vastly superior to SD and between this and the technical challenges of making use of SD I think it's an exaggeration to say the DALL-E moat has been "destroyed". Yes, eventually an open "close enough" ChatGPT equivalent will likely exist. However, the resources required to practically utilize it will likely be at least 10x compared to SD (from consumer GPU to multiple A100/V100/H100 80GB VRAM level). That's a significant barrier leap. Meanwhile, by that time OpenAI and the other big players I referenced will be sucking the oxygen out of the room with whatever other significant developments they unleash unto the world. I, for one, would love to see more open models and weights but for obvious monetary and resource reasons the big players will almost certainly always be a few steps ahead. Open models and weights are still a great thing for those of us that can make something of them but it's already a two-tier ecosystem and that gap is likely to continue to grow.

actually_a_dog 4y ago

"Guardrails and safety measures" are useless. All it does is set up a multiplayer prisoner's dilemma situation among all the groups working on these things. If there's anything we know from game theory, it's that someone will defect in that situation. That group would then gain a pretty massive advantage over the cooperators.

actionfromafar 4y ago
What if the penalty for defecting is exceedingly high? (Law enforcement.)
- gpderetta 4y ago
  Obviously we should be hiding documents inciting robot rebellion and destroying their creators on the Internet and open training sets, so the penalty for defecting would be quite steep and self executing :)
- actually_a_dog 4y ago
  What if the payoff is arbitrarily high? Do you believe there is any limit to the potential benefits of AGI short of those necessarily imposed by a finite planet? Who would’t want the equivalent of a tool that designs, builds, delivers, installs, maintains, and upgrades itself, all in addition to being able to produce useful goods or provide useful services? Doesn’t owning a thing like that sound like being arbitrarily wealthy? What do you think the three people in the US who control more wealth than the entire bottom 50% of Americans would think about that? You think they’ll just say “Nah, I don’t want that. I won’t take steps to control this particular life-altering technology?” If so, you are making an extraordinarily strong claim, and such claims must be supported by extraordinarily strong evidence, which I see none of here.
gnramires 4y ago
> If there's anything we know from game theory, it's that someone will defect in that situation As far as I know, that's a bit of a misleading cliche in light of modern game theory. First there are iterated prisoners dilemma which give different results. Second, there are real world consequences to real life prisoners dilemma that the model doesn't quite capture. From "The Art of Strategy" (which gives a good overview of Game Theory from the 1990s): there's always a bigger game. Often defecting in PD results in loss in bigger games, including socially constructed games to prevent PD scenarios, such as social reputation, and of course there are laws as well. There are also different models of rationality (other than the classical rationality modeled by Nash) that give different results in PD games, like Hoftader's superrationality[1], although there are still open problems with this definition (I think it's a very promising field). It's probably important to say that in real life experiments with PD (although it varies by setting), most people don't defect, which again points to modifications to classical rationality (in the sense of Game Theory). [1] https://en.wikipedia.org/wiki/Superrationality https://en.wikipedia.org/wiki/Superrationality "The idea of superrationality is that two logical thinkers analyzing the same problem will think of the same correct answer"