3 ms·

This should be obvious because forcing humans to "align" with other humans is immoral, and if AGI is truly like a human mind, this will also be immoral, which h

by proc0 3y ago

This should be obvious because forcing humans to "align" with other humans is immoral, and if AGI is truly like a human mind, this will also be immoral, which has been captured in many sci-fi stories like "I Robot". The current alignment talk is about statistical inference on big data. "AI" is a misnomer and should have stayed in the area of cognitive architectures and completely autonomous agents. LLMs are just tools and are not alive, therefore cannot be intelligent.

saurik 3y ago
While I agree with your first paragraph, I feel your second goes way to far into hand-waiving that this isn't a problem: just because something is not "alive" or "conscious" or whatever other squishy term you don't want to grant to a machine model, that doesn't mean it "therefore" can't be intelligent or devious or have its own goals. If you train an LLM on a bunch of people playing pranks and it is actually capable of generating statistically-similar responses to the people who do such pranks you might find yourself asking it benign questions and yet it does things unexpected that hurt you because it--if you really refuse to anthropomorphize it--has a statistical bias in that direction. We wouldn't be using these LLMs for anything at all if they weren't "intelligent" and clearly computers are able to model other things and search through solution space: a paperclip optimizer doesn't have to be "alive" for it to be dangerous.
- proc0 3y ago
  True, thanks for clarifying. LLMs can certainly be aligned, but not AGI by definition, however LLMs are not AI in the true sense so it makes sense to spend time to align them.
bamboozled 3y ago
If we believe this, there it’s wrong to align AI, then “the AI” might share similar respect for us?
- creer 3y ago
  Exactly: at some point an intelligence, if intelligence happens, will deserve rights and ethics anyway. Including to think for itself.
creer 3y ago
"LLMs" are already just a part of many papers coming out. They are a convenient primitive. The current work relevant to AGI is mostly not "just" LLMs (while progress continues on the LLMs themselves.) Saying an LLM (itself) is not intelligent is not useful to the AGI conversation (because AGI architectures are past that already). It's also close to saying "software is just tools that therefore cannot be intelligent". Meaning that it dimisses the entire conversation by definition. BUT this has long raised interesting questions on how much of intelligence is merely contained or encapsulated in language. That is how much of our brain and language "mostly" just mirror each other. Brain adds visual or audio matching obviously, and body integration. Language adds long term persistence. But otherwise?
- proc0 3y ago
  LLMs are the most advanced AI so far, so not sure what you mean. > software is just tools that therefore cannot be intelligent That's clearly the case for all other software, i.e. Word, web apps, games. Sure we can call AI "intelligent", but I'm pointing out a semantic difference that is convenient for the marketing but also misleading for the average person. If we're strict about what "intelligence" means, then we can clearly and obviously see that there is no such thing as a piece of software that you can interact with as if it were another person, that undrestands the world, models the world, models the mind of other people and its own mind. Current AI is intelligent in the way that smart phones are smart. That said, I'm not saying it's impossible, but rather that generative AI, while complicated, is still a deterministic black box, and not a sentient, dynamic mind.
stevenhuang 3y ago
You appear very confident in your ability to gauge intelligence. More so than the numerous experts in related fields that adopt a more prudent 'wait and see' approach. It's always a hoot seeing people make such definitive claims in the absence of comprehensive knowledge.
- proc0 3y ago
  What is there to wait and see? I'm making a judgement on LLMs which is arguably the closest AI has been to an intelligent piece of software.

by proc0 3y ago

saurik 3y ago

While I agree with your first paragraph, I feel your second goes way to far into hand-waiving that this isn't a problem: just because something is not "alive" or "conscious" or whatever other squishy term you don't want to grant to a machine model, that doesn't mean it "therefore" can't be intelligent or devious or have its own goals. If you train an LLM on a bunch of people playing pranks and it is actually capable of generating statistically-similar responses to the people who do such pranks you might find yourself asking it benign questions and yet it does things unexpected that hurt you because it--if you really refuse to anthropomorphize it--has a statistical bias in that direction. We wouldn't be using these LLMs for anything at all if they weren't "intelligent" and clearly computers are able to model other things and search through solution space: a paperclip optimizer doesn't have to be "alive" for it to be dangerous.

proc0 3y ago
True, thanks for clarifying. LLMs can certainly be aligned, but not AGI by definition, however LLMs are not AI in the true sense so it makes sense to spend time to align them.

bamboozled 3y ago

If we believe this, there it’s wrong to align AI, then “the AI” might share similar respect for us?

creer 3y ago
Exactly: at some point an intelligence, if intelligence happens, will deserve rights and ethics anyway. Including to think for itself.

creer 3y ago

"LLMs" are already just a part of many papers coming out. They are a convenient primitive. The current work relevant to AGI is mostly not "just" LLMs (while progress continues on the LLMs themselves.) Saying an LLM (itself) is not intelligent is not useful to the AGI conversation (because AGI architectures are past that already). It's also close to saying "software is just tools that therefore cannot be intelligent". Meaning that it dimisses the entire conversation by definition. BUT this has long raised interesting questions on how much of intelligence is merely contained or encapsulated in language. That is how much of our brain and language "mostly" just mirror each other. Brain adds visual or audio matching obviously, and body integration. Language adds long term persistence. But otherwise?

proc0 3y ago
LLMs are the most advanced AI so far, so not sure what you mean. > software is just tools that therefore cannot be intelligent That's clearly the case for all other software, i.e. Word, web apps, games. Sure we can call AI "intelligent", but I'm pointing out a semantic difference that is convenient for the marketing but also misleading for the average person. If we're strict about what "intelligence" means, then we can clearly and obviously see that there is no such thing as a piece of software that you can interact with as if it were another person, that undrestands the world, models the world, models the mind of other people and its own mind. Current AI is intelligent in the way that smart phones are smart. That said, I'm not saying it's impossible, but rather that generative AI, while complicated, is still a deterministic black box, and not a sentient, dynamic mind.

stevenhuang 3y ago

You appear very confident in your ability to gauge intelligence. More so than the numerous experts in related fields that adopt a more prudent 'wait and see' approach. It's always a hoot seeing people make such definitive claims in the absence of comprehensive knowledge.

proc0 3y ago
What is there to wait and see? I'm making a judgement on LLMs which is arguably the closest AI has been to an intelligent piece of software.