4 ms·
Ah yes, an AGI "superintelligence" piece. People who read too much scifi worry a lot about these apocalyptic AGI because they have some idea that an abstract "s
by fennecfoxen 5y ago
Ah yes, an AGI "superintelligence" piece. People who read too much scifi worry a lot about these apocalyptic AGI because they have some idea that an abstract "superintelligence" can go on self-improving exponentially using nothing but its own smartness, leaving people behind in the dust.
Curiously absent from this picture is the part where the AI is a physical thing installed somewhere. I'm sure the newborn superintelligence will immediately set about beating chip-makers like TSMC at their own game by merely imagining a better source of extreme ultraviolet for lithographic processes than the current laser-against-molten-tin solutions, to say nothing of the cleanroom manufacturing processes, the ultra-pure silicon, the various acids and even just supplies of water [1].
Meh. Worry first about firms and governments who operate computers who have good enough computers that they can spin up regular-level intelligences on demand. That's going to change more games a lot sooner and a lot faster.
[1] https://fortune.com/2021/06/12/chip-shortage-taiwan-drought-tsmc-water-usage/ https://fortune.com/2021/06/12/chip-shortage-taiwan-drought-...
- kragen 5y agoThis is the "AI in a box" argument. It's highly plausible that an AGI thus confined will find a way to escape.
- fennecfoxen 5y agoI’d worry a lot more first about a system whose leaders decide to eliminate the people, or keep them as pets, while installing many of the more efficient, compliant AIs and robots, while the AI is still comfortably inside one or more boxes. At that point does it really matter whether the tyrant who controls you is human or AI?
- kragen 5y agoProbably what you would worry more about, in a situation none of us can begin to imagine, is not a useful guide to the actual risks.