5 ms·
What you're trying to solve is something very similar to the recursive self improvement problem that MIRI[1] and friends are trying to solve in the sphere of "f
by JulianMorrison 10y ago
What you're trying to solve is something very similar to the recursive self improvement problem that MIRI[1] and friends are trying to solve in the sphere of "friendly" artificial intelligence. If something rewrites its own source code, how can you assert invariants that can be relied upon? So going and looking over there at what they've come up with may be fruitful.
[1] https://intelligence.org/ https://intelligence.org/
- grandalf 10y agoVery interesting! Looking at the papers and going to read them. Any specific papers you'd recommend?
- JulianMorrison 10y agoSorry, not qualified to have an opinion. Opening this question to the floor?
- Eliezer 10y ago"Tiling agents" would be the most arguably relevant. Provably correct agents that approve the construction of other probably correct agents obeying similar invariants.
- grandalf 10y agoThanks! Looks very interesting!
- semi-extrinsic 10y agoIs "probably" there meant to be "provably"? I assume so, but I'm not sure.
- christo 10y agoProbably.
- nialo 10y agoNote that this probably means that designing contracts of this sort of "friendly AI complete" in some sense, and therefore not a good thing to be betting on in the short term.