3 ms·

Eeeeeyeahhh. I've got to go re-read the papers, but the idea is that an AI would figure out how to approximate out the infinities, short-circuit the infinite re

by Vecr 2y ago

Eeeeeyeahhh. I've got to go re-read the papers, but the idea is that an AI would figure out how to approximate out the infinities, short-circuit the infinite regress, and figure out a theory of algorithmic similarity. The bargaining probably varies on the approximate utility function as well as the algorithm, but it's "close enough" on the scale we're dealing with. As you said, it's near useless on Earth (don't need to predict what you can control), the nearest claimed application is the various possible causal diamond overlaps between "our" ASI and various alien ASIs, where each would be unable to prevent the other from existing in a causal manner. Remember that infinite precision is an infinity too and does not really exist. As well as infinite time, infinite storage, etc. You probably don't even need infinite precision to avoid cheating on your imaginary girlfriend, just some sort of "philosophical targeting accuracy". But, you know, the only reason that's true is that everything related to imaginary girlfriends is made up.

wizzwizz4 2y ago
It doesn't matter how clever the AI is: the problem is mathematically impossible. The behaviour of some programs depends on Goldbach's conjecture. The behaviour of some programs depends on properties that have been proven independent of our mathematical systems of axioms (and it really doesn't take many bits: https://github.com/CatsAreFluffy/metamath-turing-machines https://github.com/CatsAreFluffy/metamath-turing-machines). The notion of "algorithmic similarity" cannot be described by an algorithm: the best we can get is heuristics, and heuristics aren't good enough to get TDT acausal cooperation (a high-dimensional unstable equilibrium). In practice, we can still analyse programs, because the really gnarly examples are things like program-analysis programs (see e.g. the usual proof of the undecidability of the Halting problem), and those don't tend to come up all that often. Except, TDT thought experiments posit program-analysis programs – and worse, they're analysing each other… Maybe there's some neat mathematics to attack large swathes of the solution space, but I have no reason to believe such a trick exists, and we have many reasons to believe it doesn't. (I'm pretty sure I could prove that no such trick exists, if I cared to – but I find low-level proofs like that unusually difficult, so that wouldn't be a good use of my time). > Remember that infinite precision is an infinity too and does not really exist. For finite discrete systems, infinite precision does exist. The bytestring representing this sentence is "infinitely-precise". (Infinitely-accurate still doesn't exist.)

by Vecr 2y ago

wizzwizz4 2y ago

It doesn't matter how clever the AI is: the problem is mathematically impossible. The behaviour of some programs depends on Goldbach's conjecture. The behaviour of some programs depends on properties that have been proven independent of our mathematical systems of axioms (and it really doesn't take many bits: https://github.com/CatsAreFluffy/metamath-turing-machines https://github.com/CatsAreFluffy/metamath-turing-machines). The notion of "algorithmic similarity" cannot be described by an algorithm: the best we can get is heuristics, and heuristics aren't good enough to get TDT acausal cooperation (a high-dimensional unstable equilibrium). In practice, we can still analyse programs, because the really gnarly examples are things like program-analysis programs (see e.g. the usual proof of the undecidability of the Halting problem), and those don't tend to come up all that often. Except, TDT thought experiments posit program-analysis programs – and worse, they're analysing each other… Maybe there's some neat mathematics to attack large swathes of the solution space, but I have no reason to believe such a trick exists, and we have many reasons to believe it doesn't. (I'm pretty sure I could prove that no such trick exists, if I cared to – but I find low-level proofs like that unusually difficult, so that wouldn't be a good use of my time). > Remember that infinite precision is an infinity too and does not really exist. For finite discrete systems, infinite precision does exist. The bytestring representing this sentence is "infinitely-precise". (Infinitely-accurate still doesn't exist.)