4 ms·
Meh, as long as they're not designed to improve themselves, and there are strict protocols for ensuring all active units are controllable, it's not necessarily
by gbrown 7y ago
Meh, as long as they're not designed to improve themselves, and there are strict protocols for ensuring all active units are controllable, it's not necessarily a recipe for solar system paperclipization.
- kolinko 7y agoGiven enough units any protocol will be broken by pure chance sooner or later.
- gbrown 7y agoWith appropriate organization and accounting, there's no reason malfunctioning units couldn't be repaired or reprocessed. It seems like that would be a sensible regulation for companies building self replicating machines.
- ianai 7y agoAh, but if they’re in space who’s doing the regulation?
- pavel_lishin 7y ago> there's no reason malfunctioning units couldn't be repaired or reprocessed If you have enough of them, and a long enough time scale, they might mutate in such a way to avoid being detected.
- gbrown 7y agoIt's an important concern, but I think mitigatable - especially since we're not neccessarily talking about actual self contained replicating machines, and instead more likely automated factories producing drones - a multi step process where everything has to go right for replication, and into which safeguards can be engineered/studied. Basically, the process needs to be designed to fail due to entropy without external intervention/maintenance. If we become advanced enough to consider building such things, we can engage in a serious investigation of the risks - for now it's just speculation.