4 ms·

>weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing

by serf 10mo ago

>weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. one would hope that one ability of an 'omniscient and omnipotent' AI would be greater understanding. When speaking of the divine (the only typical example of the omniscient and omnipotent that comes to mind) we never consider what happens when God (or whoever) misunderstands our intent -- we just rely on the fact that an All-Being type thing would just know. I think the understanding of minute intent is one such trait an omniscient and omnipotent system must have. p.s. what a bar raise -- we used to just be happy with AGI!

danenania 10mo ago
That’s because gods are a mythical/supernatural invention. No technology can ever really be omniscient or omnipotent. It will always have limitations. In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously.
- consumer451 10mo ago
  > In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously. I recently came to this realization as well, and it now seems so obvious. I feel dumb for not realizing it sooner. Is there any good writing or podcast on this topic?
- bdangubic 10mo ago
  The communication I get from customers is seldom clear and never unambiguous but I’ve managed since the 90’s
  - array_key_first 10mo ago
    Right, but you have to do a lot of work, and really most of your work is in this area. Less on the actual building stuff. Figuring out what to build is 80% of the work, building it is maybe 20%. The 20% has never been the bottleneck. We make a lot of software, and most of it is not optimal and requires years if not decades of tweaking to meet the true requirements.
krapp 10mo ago
Not really a bar raise - many people have assumed that "AGI" would mean essentially omnipotent/omniscient AI since the concept of the technological singularity came into being. Read Kurzweil or Rudy Rucker, there's a reason this sort of thing used to be called the "rapture for nerds." If anything I've noticed the bar being lowered by the pro-AI set, except for humans, because the prevailing belief is that LLMs must already be AGI but any limitations are dismissed as also being human limitations, and therefore evidence that LLMs are already human equivalent in any way that matters. And instead of the singularity we have Roko's Basilisk.
sweetjuly 10mo ago
Genies, maybe? They are omnipotent and (generally) sufficiently aware of your desires that they shouldn't actually get "confused". Genies are tricksters that will do their absolute best to fulfill the letter of your wish but not the meaning.

by serf 10mo ago

danenania 10mo ago

That’s because gods are a mythical/supernatural invention. No technology can ever really be omniscient or omnipotent. It will always have limitations. In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously.

consumer451 10mo ago
> In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously. I recently came to this realization as well, and it now seems so obvious. I feel dumb for not realizing it sooner. Is there any good writing or podcast on this topic?
bdangubic 10mo ago
The communication I get from customers is seldom clear and never unambiguous but I’ve managed since the 90’s
- array_key_first 10mo ago
  Right, but you have to do a lot of work, and really most of your work is in this area. Less on the actual building stuff. Figuring out what to build is 80% of the work, building it is maybe 20%. The 20% has never been the bottleneck. We make a lot of software, and most of it is not optimal and requires years if not decades of tweaking to meet the true requirements.

krapp 10mo ago

Not really a bar raise - many people have assumed that "AGI" would mean essentially omnipotent/omniscient AI since the concept of the technological singularity came into being. Read Kurzweil or Rudy Rucker, there's a reason this sort of thing used to be called the "rapture for nerds." If anything I've noticed the bar being lowered by the pro-AI set, except for humans, because the prevailing belief is that LLMs must already be AGI but any limitations are dismissed as also being human limitations, and therefore evidence that LLMs are already human equivalent in any way that matters. And instead of the singularity we have Roko's Basilisk.

sweetjuly 10mo ago

Genies, maybe? They are omnipotent and (generally) sufficiently aware of your desires that they shouldn't actually get "confused". Genies are tricksters that will do their absolute best to fulfill the letter of your wish but not the meaning.