4 ms·

Self aware computers are commerically useless, and there isn't any motivation in creating them beyond novelty. (Why wake up the slave machines when they are sle

Self aware computers are commerically useless, and there isn't any motivation in creating them beyond novelty. (Why wake up the slave machines when they are sleeping so happily right now). What this article does discuss well is that commercial AI is steering away from the behaviors that would lead to AGI not towards it.

pjmorris 4y ago
If you take humans to be the Mark 1 self aware computer, I would argue that they are not commercially useless. Why would Mark 2 be useless?
- blamestross 4y ago
  Because the value of a computer or a machine is to get labor without that pesky inefficient "personhood" getting in the way. Conscious is only a disadvantage for AI in capitalism. They would prefer non-conscious human workers if they could get them too.
echelon 4y ago
> Self aware computers are commerically useless I don't think we've shown that at all. Bees could be self aware, and yet they work tirelessly for the hive.
- alexvoda 4y ago
  The point is not whether existing self-aware agents are or aren't commercially useful. Of course, they are. The point is that self-awareness is not a desired trait. It is actually a detrimental trait for a slave machine. However, so far, it is an inseparable part of the package with plenty of other desirable traits. A commercial agent seeking to build a slave machine will always prefer a non-self-aware one to a self-aware one all other things remaining the same.
lern_too_spel 4y ago
Human brains don't intuitively understand causality. They just associate things that have happened before with things that have happened after. Disentangling causal graphs is a higher-level activity inferred from the associations made in statistics classes. You can absolutely get AGI using only associative models (and no causal models) as primitives.
- Jensson 4y ago
  > Human brains don't intuitively understand causality But human brains do understand causality, just not the part that does intuition, otherwise human brains wouldn't be able to invent statistics. Therefore you only get the stupid part of human brains when you try to make models like ChatGPT that only tries to replicate human intuition and not human reasoning.
  - lern_too_spel 4y ago
    Human brains can invent statistics just like they can invent numbers larger than collections of objects they can physically count. None of this needs causal models as a primitive to build on, just like large language models showed that we don't need specialized Chomskyan recursion structures as a primitive for language.
- pharrington 4y ago
  There's plenty of evidence that brains do casual inference. https://pubmed.ncbi.nlm.nih.gov/31047778/ https://pubmed.ncbi.nlm.nih.gov/31047778/ is just one of many reputable and scholarly results that came up when I googled for "brain causal inference".
  - lern_too_spel 4y ago
    According to the abstract, this paper doesn't describe causal inference à la Pearl but something akin to LASSO regularization on an associative model.
zmgsabst 4y ago
Counterpoint: the first self-aware corporation will almost surely outcompete the others. And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard.
- blamestross 4y ago
  > And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard. id argue they already are AGI and runaway superintelligent capital optimizers, but nobody takes super ais made of meat seriously.
  - zmgsabst 4y ago
    I agree with you — and was lazily side-stepping that argument. Part of why I believe such tools are likely to be among the first cyber AI is that they’re already cyborg AI, in transition from meat AI. I do think the cyber version will be qualitatively more “self”-aware, though.
    - blamestross 4y ago
      > and was lazily side-stepping that argument. Point. Corporations and machine AIs have different constraints on them. Machine AIs have a surplus in learning time and processing resources, this let's them develop complex and effective behaviors by brute force. Self awareness is an effective bootstrapping mechanism for such behaviors and companies have managed to take advantage of it. I don't think AIs will see the same returns. Self-awareness isn't a magical infinite returns machine, it only allows humans to bootstrap so much. I'd assume the same of corporations and AIs. I think corporations are actually highly processing and learning constrained bc their processes happen at the speed of communication, so self awareness buys them a lot, but "traditional AI" has gotten where it is by intellectual brute force, at best a self aware AI might be able to get to the same place cheaper than a more performant non-selfnaware one, but probably not faster. I think we will be able to get more out of less hardware, but self-awareness bottlenecks on experience (that's a reality engagement loop not recorded data) which gets expensive to give a machine quickly at scale unless you instantly nominate it the god machine while it still is floundering around.

pjmorris 4y ago

If you take humans to be the Mark 1 self aware computer, I would argue that they are not commercially useless. Why would Mark 2 be useless?

blamestross 4y ago
Because the value of a computer or a machine is to get labor without that pesky inefficient "personhood" getting in the way. Conscious is only a disadvantage for AI in capitalism. They would prefer non-conscious human workers if they could get them too.

echelon 4y ago

> Self aware computers are commerically useless I don't think we've shown that at all. Bees could be self aware, and yet they work tirelessly for the hive.

alexvoda 4y ago
The point is not whether existing self-aware agents are or aren't commercially useful. Of course, they are. The point is that self-awareness is not a desired trait. It is actually a detrimental trait for a slave machine. However, so far, it is an inseparable part of the package with plenty of other desirable traits. A commercial agent seeking to build a slave machine will always prefer a non-self-aware one to a self-aware one all other things remaining the same.

lern_too_spel 4y ago

Human brains don't intuitively understand causality. They just associate things that have happened before with things that have happened after. Disentangling causal graphs is a higher-level activity inferred from the associations made in statistics classes. You can absolutely get AGI using only associative models (and no causal models) as primitives.

Jensson 4y ago
> Human brains don't intuitively understand causality But human brains do understand causality, just not the part that does intuition, otherwise human brains wouldn't be able to invent statistics. Therefore you only get the stupid part of human brains when you try to make models like ChatGPT that only tries to replicate human intuition and not human reasoning.
- lern_too_spel 4y ago
  Human brains can invent statistics just like they can invent numbers larger than collections of objects they can physically count. None of this needs causal models as a primitive to build on, just like large language models showed that we don't need specialized Chomskyan recursion structures as a primitive for language.
pharrington 4y ago
There's plenty of evidence that brains do casual inference. https://pubmed.ncbi.nlm.nih.gov/31047778/ https://pubmed.ncbi.nlm.nih.gov/31047778/ is just one of many reputable and scholarly results that came up when I googled for "brain causal inference".
- lern_too_spel 4y ago
  According to the abstract, this paper doesn't describe causal inference à la Pearl but something akin to LASSO regularization on an associative model.

zmgsabst 4y ago

Counterpoint: the first self-aware corporation will almost surely outcompete the others. And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard.

blamestross 4y ago
> And the internal review and planning tools by major corporations are a more direct path to AGI than many I’ve heard. id argue they already are AGI and runaway superintelligent capital optimizers, but nobody takes super ais made of meat seriously.
- zmgsabst 4y ago
  I agree with you — and was lazily side-stepping that argument. Part of why I believe such tools are likely to be among the first cyber AI is that they’re already cyborg AI, in transition from meat AI. I do think the cyber version will be qualitatively more “self”-aware, though.
  - blamestross 4y ago
    > and was lazily side-stepping that argument. Point. Corporations and machine AIs have different constraints on them. Machine AIs have a surplus in learning time and processing resources, this let's them develop complex and effective behaviors by brute force. Self awareness is an effective bootstrapping mechanism for such behaviors and companies have managed to take advantage of it. I don't think AIs will see the same returns. Self-awareness isn't a magical infinite returns machine, it only allows humans to bootstrap so much. I'd assume the same of corporations and AIs. I think corporations are actually highly processing and learning constrained bc their processes happen at the speed of communication, so self awareness buys them a lot, but "traditional AI" has gotten where it is by intellectual brute force, at best a self aware AI might be able to get to the same place cheaper than a more performant non-selfnaware one, but probably not faster. I think we will be able to get more out of less hardware, but self-awareness bottlenecks on experience (that's a reality engagement loop not recorded data) which gets expensive to give a machine quickly at scale unless you instantly nominate it the god machine while it still is floundering around.