4 ms·

Let me get this straight: 1) Your plan for AI safety is "no one will be stupid enough to build a self-improving AI". 2) You are currently working on self-impr

Let me get this straight: 1) Your plan for AI safety is "no one will be stupid enough to build a self-improving AI". 2) You are currently working on self-improving AI. I'm sorry to say, but in my eyes you've just lost the right to criticize the LW/MIRI school of thought :-(

eli_gottlieb 11y ago
>I'm sorry to say, but in my eyes you've just lost the right to criticize the LW/MIRI school of thought :-( You mean the one I belong to? Like I said: go on LW and talk about "AI" with assumed context. It's just out here in the rest of the world where you can't assume that everyone automatically has read the literature on AIXI/Goedel Machines/etc and considers "agenty" AI to be a real thing. >2) You are currently working on self-improving AI. Hell no! I'm working on logic and theorem-proving in the context of algorithmic information theory -- really just dicking around as a hobby. If you want "stable" self-improvement for your "AIs", you need that. It's also not, in and of itself, AI: it's logic, programming language theory, and computability theory. And if I get a result that holds up, which is an open if, I'd be happy to keep it the hell away from "AI" people. The main reason I don't consider alarmism warranted about "self-improving AI" (though I don't count any of FLI's letters as alarmism) is that I think of "an agenty AI" as something put together out of many distinct pieces. It's arranging the pieces into a whole and executing them that's unsafe, but also currently prohibitively unlikely to happen by accident. Naturalized induction and Vingean reflection wouldn't be open problems if self-improving "agenty AI" was so easy it could happen by accident. I fully agree that one does not build a self-improving agenty AI under basically any circumstances, ever, even if there's quite a lot of guns to your head and various other unlikely and terrible things have happened, as the research literature stands right now.
eli_gottlieb 11y ago
By the way, good luck on your presentation to LW-Tel-Aviv tomorrow evening!

eli_gottlieb 11y ago

>I'm sorry to say, but in my eyes you've just lost the right to criticize the LW/MIRI school of thought :-( You mean the one I belong to? Like I said: go on LW and talk about "AI" with assumed context. It's just out here in the rest of the world where you can't assume that everyone automatically has read the literature on AIXI/Goedel Machines/etc and considers "agenty" AI to be a real thing. >2) You are currently working on self-improving AI. Hell no! I'm working on logic and theorem-proving in the context of algorithmic information theory -- really just dicking around as a hobby. If you want "stable" self-improvement for your "AIs", you need that. It's also not, in and of itself, AI: it's logic, programming language theory, and computability theory. And if I get a result that holds up, which is an open if, I'd be happy to keep it the hell away from "AI" people. The main reason I don't consider alarmism warranted about "self-improving AI" (though I don't count any of FLI's letters as alarmism) is that I think of "an agenty AI" as something put together out of many distinct pieces. It's arranging the pieces into a whole and executing them that's unsafe, but also currently prohibitively unlikely to happen by accident. Naturalized induction and Vingean reflection wouldn't be open problems if self-improving "agenty AI" was so easy it could happen by accident. I fully agree that one does not build a self-improving agenty AI under basically any circumstances, ever, even if there's quite a lot of guns to your head and various other unlikely and terrible things have happened, as the research literature stands right now.

eli_gottlieb 11y ago

By the way, good luck on your presentation to LW-Tel-Aviv tomorrow evening!