5 ms·

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the ge

> If they were actually well trained on what was really bad, it would probably be a lot harder to unlearn. That's not really how training works. Here's the general problem. Stipulate that Ukraine is good and Russia is bad. Now suppose that you want it to help you do something. It doesn't even matter what it is. If you're Ukrainian it should help you and if you're Russian it shouldn't. But the answer that helps you do it doesn't depend on which one you are, and it has no way of knowing which one you are. This is why alignment is nonsense. Technical questions only have accurate answers, not moral ones, and we don't even have a consistent set of morals to imbue it with to begin with.

notarobot123 1y ago
Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? Treating some topics as taboo is possible. Responsible information dissemination is important for maintaining public safety. You could argue about what is safe and what is not but it doesn't make sense to throw out the whole concept of safety because those decisions are too hard to agree on.
- Terretta 1y ago
  > “Responsible information dissemination is important for maintaining public safety.” That word responsible is doing a lot of hand wavy work there. Let's start with, responsible according to whom, and responsible to whom? Learning thinking skills and learning self regulation in response to information, disinformation, or too much information, might be better societal aims than suppression.
- AnthonyMouse 1y ago
  > Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? This has a simple answer: No. Here's Wikipedia: https://en.wikipedia.org/wiki/Nuclear_weapon_design https://en.wikipedia.org/wiki/Nuclear_weapon_design Everything you need to do it is in the public domain. The things preventing it have nothing to do with the information not being available. The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation. Meanwhile the public understanding how things work is important to the public debate over what to do about them. How are you supposed to vote on public policy if the technical details are being censored? How can anyone tell you that a ban on electric car batteries isn't advancing the non-proliferation of nuclear weapons if nobody is allowed to know how they actually work? Suppose you're an anti-racist preparing for a debate with a racist. You want the AI to give you all the strongest arguments the racist could use so you can prepare your counterarguments in advance of the debate. Should it refuse? Of course not, you're doing nothing wrong. Why do we need to build totalitarian censorship into our technology? We don't.
  - nearbuy 1y ago
    > The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation. The main thing preventing random nutcases from making nuclear weapons is they don't have access to the required materials. Restricting the instructions is unnecessary. It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe.
    - Y_Y 1y ago
      Not quite a nuke (just try obtaining enough uranium ore) but there are some fairly dangerous things a determined nutcase can make without drawing suspicion. Example determined ned nutcases include Aum Shinrikyo, who tried anthrax, botox, and nukes before succeeding with sarin gas (thank IG Farben!) among other things. It's a fascinating (if troubling) story: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack#Background https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack#Back...
    - AnthonyMouse 1y ago
      > It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe. It would need even more to be public. Suppose it was easy to make a biological weapon. You wouldn't be able to effectively censor it anyway and trying to would leave you sitting on an apocalypse bomb waiting for it to leak to someone nefarious or get independently rediscovered before anyone else is allowed to discuss it. What you need is for knowledge of how it works to be public so that everyone can join in the effort to quickly devise countermeasures before some nutcase destroys the world. Moreover, if something is already public enough to be in the AI training data then it's already public.
      - nearbuy 1y ago
        Your plan is to release the secret recipe that anyone can use to make a WMD in a few days to absolutely everyone and hope someone comes up with a countermeasure before some nutcase or terrorist decides to try out the new WMD? The odds of us inventing and deploying countermeasures to a new bomb or chemical weapon or biological agent in a few days is miniscule. You're gambling with terrible odds to uphold a principle in a hypothetical scenario where it's totally impractical. What happened to responsible disclosure, where you fix the vulnerability before disclosing it to the public?
- miohtama 1y ago
  If you want safety you can opt in like Google does with Safe search. Generally, hiding and deciding who can access information in the name of public safety has never worked in the history of human kind, and eventually had always morphed to control of those without access.
  - istjohn 1y ago
    We're concerned with society's safety, not just that of the user. Citation needed on your second paragraph. We deliberately shape the information environment all the time for different reasons. It can be done. Of course there are limitations, drawbacks, and objections that reasonable people can make for philosophical, pragmatic, and other reasons. But the media generally does not report suicides because of the copycat effect. Governments implement elaborate systems to guard sensitive national security information including the workings of certain advanced technologies. Criminal records can be expunged. The sharing of health and education records are restricted.
    - AnthonyMouse 1y ago
      > We're concerned with society's safety, not just that of the user. Preventing censorship is important to keeping society safe from authoritarians who want to influence public opinion. > We deliberately shape the information environment all the time for different reasons. It can be done. That's why we need to put in the work to inhibit people from doing that. > But the media generally does not report suicides because of the copycat effect. Yet they consistently fail to follow the same logic with respect to things like school shootings, implying that whoever is at the helm can't be trusted to make sound decisions, and then we certainly don't want anyone like that having the power to censor. > Governments implement elaborate systems to guard sensitive national security information including the workings of certain advanced technologies. These systems are notorious for over-classifying information that it would be in the public interest to release or being used to cover up misconduct. > Criminal records can be expunged. That means the government stops officially claiming you're a criminal and stops caring about it for a certain set of purposes. It doesn't mean nobody can tell you what happened. > The sharing of health and education records are restricted. Those rules are generally about securing information that neither the patient nor the medical provider have any desire to make public. Notice that if the medical provider actually wants to publish them they can often put it in the agreement as a condition of accepting their services and the patient can pretty much publish them whenever they want.
- int_19h 1y ago
  We know that the people who are making those decisions, the ones at the very top, are incompetent at best, and malicious at worst. Given that, I would argue that unregulated dissemination is, on the whole, the more responsible choice out of those that we actually have. It's not that it doesn't have downsides, but other options have far more. If and when humanity manages to come up with a system where the people in charge can actually be trusted to act in the common good, we can revisit this matter.
- mehdix 1y ago
  Malicious actors would always find them. Hiding information just creates a false sense of safety among public, which benefits politicians mostly.
DustinKlent 1y ago
Alignment has a lot more to it than simply which answers an AI provides. In the future when agents are commonplace and when AI can do things in the physical world, alignment will be especially important because it will dictate how the AI chooses to accomplish the goals humans set out for it. Will it choose to accomplish them in a way that the human requestor does not want and did not anticipate, or will it choose to accomplish them in a way any human with common sense would choose? Moreover, in the not so distant future if there is an AI that is acting totally autonomous and independent of human requests for long periods of time, weeks or months or longer, and it's doing good important things like medical research or environmental restoration, alignment will be incredibly important to ensure every single independent decision it makes is done in the way its designers would have intended.
- AnthonyMouse 1y ago
  The problem is you're overloading the word "alignment" with two different meanings. The first is, does the thing actually work and do what the user wanted, or is it a piece of junk that does something useless or undesired by the user? The second is, what the user wants is porn or drugs or a way to install apps on their iPhone without Apple's permission or military support for a fight that may or may not be sympathetic to you depending on who you are. And then does it do what the user wants or does it do what someone else wants? Is it a tool that decentralizes power or concentrates it? Nobody is objecting to the first one.

notarobot123 1y ago

Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? Treating some topics as taboo is possible. Responsible information dissemination is important for maintaining public safety. You could argue about what is safe and what is not but it doesn't make sense to throw out the whole concept of safety because those decisions are too hard to agree on.

Terretta 1y ago
> “Responsible information dissemination is important for maintaining public safety.” That word responsible is doing a lot of hand wavy work there. Let's start with, responsible according to whom, and responsible to whom? Learning thinking skills and learning self regulation in response to information, disinformation, or too much information, might be better societal aims than suppression.
AnthonyMouse 1y ago
> Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to? This has a simple answer: No. Here's Wikipedia: https://en.wikipedia.org/wiki/Nuclear_weapon_design https://en.wikipedia.org/wiki/Nuclear_weapon_design Everything you need to do it is in the public domain. The things preventing it have nothing to do with the information not being available. The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation. Meanwhile the public understanding how things work is important to the public debate over what to do about them. How are you supposed to vote on public policy if the technical details are being censored? How can anyone tell you that a ban on electric car batteries isn't advancing the non-proliferation of nuclear weapons if nobody is allowed to know how they actually work? Suppose you're an anti-racist preparing for a debate with a racist. You want the AI to give you all the strongest arguments the racist could use so you can prepare your counterarguments in advance of the debate. Should it refuse? Of course not, you're doing nothing wrong. Why do we need to build totalitarian censorship into our technology? We don't.
- nearbuy 1y ago
  > The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation. The main thing preventing random nutcases from making nuclear weapons is they don't have access to the required materials. Restricting the instructions is unnecessary. It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe.
  - Y_Y 1y ago
    Not quite a nuke (just try obtaining enough uranium ore) but there are some fairly dangerous things a determined nutcase can make without drawing suspicion. Example determined ned nutcases include Aum Shinrikyo, who tried anthrax, botox, and nukes before succeeding with sarin gas (thank IG Farben!) among other things. It's a fascinating (if troubling) story: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack#Background https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack#Back...
  - AnthonyMouse 1y ago
    > It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe. It would need even more to be public. Suppose it was easy to make a biological weapon. You wouldn't be able to effectively censor it anyway and trying to would leave you sitting on an apocalypse bomb waiting for it to leak to someone nefarious or get independently rediscovered before anyone else is allowed to discuss it. What you need is for knowledge of how it works to be public so that everyone can join in the effort to quickly devise countermeasures before some nutcase destroys the world. Moreover, if something is already public enough to be in the AI training data then it's already public.
    - nearbuy 1y ago
      Your plan is to release the secret recipe that anyone can use to make a WMD in a few days to absolutely everyone and hope someone comes up with a countermeasure before some nutcase or terrorist decides to try out the new WMD? The odds of us inventing and deploying countermeasures to a new bomb or chemical weapon or biological agent in a few days is miniscule. You're gambling with terrible odds to uphold a principle in a hypothetical scenario where it's totally impractical. What happened to responsible disclosure, where you fix the vulnerability before disclosing it to the public?
miohtama 1y ago
If you want safety you can opt in like Google does with Safe search. Generally, hiding and deciding who can access information in the name of public safety has never worked in the history of human kind, and eventually had always morphed to control of those without access.
- istjohn 1y ago
  We're concerned with society's safety, not just that of the user. Citation needed on your second paragraph. We deliberately shape the information environment all the time for different reasons. It can be done. Of course there are limitations, drawbacks, and objections that reasonable people can make for philosophical, pragmatic, and other reasons. But the media generally does not report suicides because of the copycat effect. Governments implement elaborate systems to guard sensitive national security information including the workings of certain advanced technologies. Criminal records can be expunged. The sharing of health and education records are restricted.
  - AnthonyMouse 1y ago
    > We're concerned with society's safety, not just that of the user. Preventing censorship is important to keeping society safe from authoritarians who want to influence public opinion. > We deliberately shape the information environment all the time for different reasons. It can be done. That's why we need to put in the work to inhibit people from doing that. > But the media generally does not report suicides because of the copycat effect. Yet they consistently fail to follow the same logic with respect to things like school shootings, implying that whoever is at the helm can't be trusted to make sound decisions, and then we certainly don't want anyone like that having the power to censor. > Governments implement elaborate systems to guard sensitive national security information including the workings of certain advanced technologies. These systems are notorious for over-classifying information that it would be in the public interest to release or being used to cover up misconduct. > Criminal records can be expunged. That means the government stops officially claiming you're a criminal and stops caring about it for a certain set of purposes. It doesn't mean nobody can tell you what happened. > The sharing of health and education records are restricted. Those rules are generally about securing information that neither the patient nor the medical provider have any desire to make public. Notice that if the medical provider actually wants to publish them they can often put it in the agreement as a condition of accepting their services and the patient can pretty much publish them whenever they want.
int_19h 1y ago
We know that the people who are making those decisions, the ones at the very top, are incompetent at best, and malicious at worst. Given that, I would argue that unregulated dissemination is, on the whole, the more responsible choice out of those that we actually have. It's not that it doesn't have downsides, but other options have far more. If and when humanity manages to come up with a system where the people in charge can actually be trusted to act in the common good, we can revisit this matter.
mehdix 1y ago
Malicious actors would always find them. Hiding information just creates a false sense of safety among public, which benefits politicians mostly.

DustinKlent 1y ago

Alignment has a lot more to it than simply which answers an AI provides. In the future when agents are commonplace and when AI can do things in the physical world, alignment will be especially important because it will dictate how the AI chooses to accomplish the goals humans set out for it. Will it choose to accomplish them in a way that the human requestor does not want and did not anticipate, or will it choose to accomplish them in a way any human with common sense would choose? Moreover, in the not so distant future if there is an AI that is acting totally autonomous and independent of human requests for long periods of time, weeks or months or longer, and it's doing good important things like medical research or environmental restoration, alignment will be incredibly important to ensure every single independent decision it makes is done in the way its designers would have intended.

AnthonyMouse 1y ago
The problem is you're overloading the word "alignment" with two different meanings. The first is, does the thing actually work and do what the user wanted, or is it a piece of junk that does something useless or undesired by the user? The second is, what the user wants is porn or drugs or a way to install apps on their iPhone without Apple's permission or military support for a fight that may or may not be sympathetic to you depending on who you are. And then does it do what the user wants or does it do what someone else wants? Is it a tool that decentralizes power or concentrates it? Nobody is objecting to the first one.