3 ms·
So what happens when the world becomes hyper optimized with closed loop AI agents recursively trying to optimize everything deemed sub optimal?
by nickandbro 4mo ago
So what happens when the world becomes hyper optimized with closed loop AI agents recursively trying to optimize everything deemed sub optimal?
- peheje 4mo agothere will be a lot of paper clips
- simianwords 4mo agoOften repeated meme doesn’t have any bearing to reality. The orthogonality thesis sounds like a fun gotcha but if you give it some thought you realise how strange it sounds and the opposite thesis - collinearity thesis is actually correct. 1. Intelligence transfers and compounds 2. Goals of agents are not arbitrary 3. Our goals and agent goals are more likely to be aligned at the deeper level
- IsTom 4mo agoWhen primate family produced a super-primate intelligence it sure aligned with the good of all of them.
- deleted 4mo ago[deleted]
- mofeien 4mo agoI would assume that shortly after, the solar system will be hyper optimized as well, then the milky way, then the local cluster, and so on. Everything will be close to optimal afterwords, and I sure hope we will have specified the target function for that optimization correctly in the single attempt that we will have had.
- Readerium 4mo agoLoll
- Groxx 4mo agoGithub outages will probably get worse.
- deleted 4mo ago[deleted]
- layer8 4mo agoIf it optimizes itself away because it’s suboptimal, that wouldn’t be the worst outcome. ;)