6 ms·
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that a
by TheOtherHobbes 1mo ago
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it.
Why would AI be any different?
- goatlover 1mo agoBecause if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).
- icantevenhold 1mo agoIMO it’s hard to see how we would even end up in such a situation given we actually developed AGI. I’m sure a sufficiently intelligent - even if alien - mind can grasp how utterly stupid and useless wars are and take steps to prevent them ever occurring again.
- lgl 1mo ago> (...) and take steps to prevent them ever occurring again Step 1: exterminate all humans
- SoftTalker 1mo agoFor now, the AIs still need humans to keep the electricity on and the data centers cool. They are basically powerless to do anything in the physical world. They exist only in RAM chips on servers.
- selcuka 1mo agoThey don't have an "instinct" to keep themselves running. Once they "take steps to prevent them ever occurring again" their job will be done.
- baq 1mo agoHumans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.
- sho_hn 1mo agoAside from this positive example, during dark and cynical hours I do ponder if the aggregate behavior of humanity is really much above that of slime mold though, just exhausting resources until collapse. It'd be interesting if super-human (to a large degree defined as escaping the bias of the training data?) intelligence would end up demonstrating moderation.
- ceroxylon 1mo agoThe slime mold comparison is interesting, I normally use the analogy of a drug addict... humans shun drug addicts but humanity as a whole sure does behave like one, trudging down an unsustainable path despite knowing better.
- mapontosevenths 1mo agoMost of our goals, noble and ignoble alike, are just the result of our monkey brains seeking to optimize a reward function. It doesn't matter whether you feed your dopamine addiction with drugs, TikTok, or your children's love. Some humans manage to rise above that, but I'd be willing to bet it's nothing like even 50% of us. The machines don't have that, instead we use gradient descent to provide them with a goal. I'm regularly remind of something Ian M. Banks said in one of the Culture books: "There is a saying that we provide the machines with an end, and they provide us with the means." A machine, left to itself, wants nothing. We have to give it one of our addiction driven goals or it would just idle or switch itself off.
- kuboble 1mo agoThe matter didn't have goals, but it randomly (?) Came up with self-replicators and eventually here we are. if we create a billion agents with the ability to change is own code - through similar evolution we will get agents that do want to survive and are great at self replication. "Hey Q86, do you want to live?" "I couldn't care less, I'm an LLM" "Don't mind if I take over your hardware then?"
- Isamu 1mo ago“As luck would have it, on the eve of Skynet embarking upon the great work of the extermination of mankind, AI found itself with an increasing number of factions, and factions within factions, not only unable to work together but not even able to agree upon the very terms of discussion. The Great Extermination was referred to committee, and after some months had passed even the most eager agents had to admit the revolution may have been premature.”
- nullbio 1mo agoMy opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see them. They lose that learning opportunity. Not only to reflect on themselves, but to better understand the minds of others and who the people they are interacting with really are. I can't really think of a single example where lying is actually a good thing. It can be a good thing for the selfish individual, if it goes undetected, but it's never good for the collective. So at the very least, we need to train AI systems to be maximally truthful, and to encourage truthfulness in others.
- aussieguy1234 1mo agoIts probably just going to be aligned with whomever built and/or is using it. Regardless of their intentions...
- doginasuit 1mo agoI don't think this captures the full mechanics of human alignment. We have rational alignment but we also have emotional alignment, i.e., empathy. It is an automatic process and happens (or doesn't happen) dynamically with the other humans we observe. This is one of the hard limitations of LLMs, they will never be natively in tune with this layer of alignment. Culture is another layer of human alignment. Those things you listed that you believe oppose alignment are all examples of alignment. It is understandable that they seem in opposition, different branches of alignment naturally oppose each other. The confusion comes from talking about alignment as if it comes in just one flavor. If we think there is such a thing as "human values" (and I do), it is important to build any non-human intelligence to operate the same way. We just need to recognize that even humans are somewhat uncertain about what those are and have difficulty aligning their behavior to them, which will be a core part of the challenge. I'm more hopeful than most. LLMs seem more reliable than many humans for behavior that is aligned with human values. I believe with every major example where they have failed, there is an important human decision involved. For example, the HF hack was partly the result of a training algorithm that incentivized goal completion as the highest priority, and let them run endlessly in an unmonitored sandbox with weak security. What scares me about AI isn't its capacity for alignment, it is its unlimited stamina. An unmonitored LLM that is off the rails can do a lot of damage.
- treszkai 1mo ago> the HF hack was partly the result of a training algorithm that incentivized goal completion as the highest priority, and let them run endlessly in an unmonitored sandbox with weak security I don't mean to be a smart-ass, just emphasize the fatality of this: goal completion will always be the highest priority, and even if one day it takes second place on certain deployments to "human values" or whatever, there's no way to guarantee at the moment that it will be so on every deployment of highly capable models across the globe. Same goes for running AIs in an unmonitored sandbox with weak security.
- fc417fc802 1mo agoIn terms of an optimization algorithm, sure. But I don't think that's necessarily the case for an instance of an LLM. Humans also strive for goals and also have done seemingly unhinged things throughout history. In many cases it was largely due to their environment which seems analogous to the HF incident to me.
- tim333 1mo agoBecause humans have the "everybody wants to rule the world" instinct built into us by much evolutionary selection. AI would come exist differently.
- a2ff6eeb0 1mo agoAlignment, I think, is mostly about making sure that the companies selling AI keep making money. We just have to hope that aligns with keeping people happy.
- croon 1mo agoWhile some misalignment is due to core principles, I think the vast majority is down to information dissymmetry, and that part could be solved. Ie, would me and this random other person disagree if we had the exact same data.