3 ms·
Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom see
by DalasNoin 3y ago
Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited?
Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.
- NoOn3 3y agoQ* reminds me of Q-learning and A* search, maybe it's somehow related to combination of such algorithms...
- golol 3y agoThere is foom as in decently fast self improvement to super human level, but that is far from a dangerous foom scenario where that AI then escapes. You can just turn it off. It can not spread through the internet or something like that, that seems extremely unrealistic to me.
- pixl97 3y agoI'm not worried about foom at the moment because of the scale/power required. The following is a bigger potential risk in my eyes. We keep gradually growing AI ability while rapidly growing compute capacity and miniaturization. Then in 10-20 years when we have data center capabilities in the size of a cellphone/laptop we have a massive AI algorithm improvement that brings ASI to something portable. How can you turn off something that can be ran on one of the hundreds of millions of computers already out there.