2 ms·

Other commenters have mentioned that these particular points have been refuted elsewhere, so I won't rehash that. But I do think it's interesting that the autho

Other commenters have mentioned that these particular points have been refuted elsewhere, so I won't rehash that. But I do think it's interesting that the author frames the "orthodox" position as the one taking AI alignment seriously. That's pretty big progress and I think the folks who have been sounding the alarm[0] about AGI should give themselves a pat on the back that the zeitgeist has shifted so much. The arguments from the anti-alarmism camp have been so consistently weak, I think the best explanation for their motivation is still something like "All this AGI stuff sounds so sci-fi. It can't be a real concern". It just goes to show how strongly we bias our expectations of the future on our what we've personally experienced. You really have to try hard to escape that default mode of thinking. [0] By sounding the alarm I mean saying things like "This is at least a possible scenario with huge error bars on when it might happen, maybe at least some people should be thinking hard about this, rather than nobody"

batavian 5y ago
It doesn't really even sound sci-fi, or rather, it would make for some bad sci-fi. What we are at risk of, is having deterministic model artifacts trigger complex unintended consequences from being fetishized as decision-making agents. Considering how few CS and AI people I've met who even know that e.g. pragmatics are a critical component of sentence meaning, I sense the most proximate AI dystopia is a lot closer to this latter scenario. Conceptually the possibility of an AGI does present a real challenge (& one worth grappling); in practice, we haven't made any progress really toward realizing that problem. We've become good at feeding "large" datasets into algorithms that are many orders of magnitude less advanced than a lizard brain. Then we get to the issue that our large datasets are only large when thinking of classical statistical sample sizes. They nonetheless are lacking in redundancy, interactivity, stakes or any centered perspective such that there is no chance of anything resembling intelligence, agency, or plain old intentions arising.

batavian 5y ago

It doesn't really even sound sci-fi, or rather, it would make for some bad sci-fi. What we are at risk of, is having deterministic model artifacts trigger complex unintended consequences from being fetishized as decision-making agents. Considering how few CS and AI people I've met who even know that e.g. pragmatics are a critical component of sentence meaning, I sense the most proximate AI dystopia is a lot closer to this latter scenario. Conceptually the possibility of an AGI does present a real challenge (& one worth grappling); in practice, we haven't made any progress really toward realizing that problem. We've become good at feeding "large" datasets into algorithms that are many orders of magnitude less advanced than a lizard brain. Then we get to the issue that our large datasets are only large when thinking of classical statistical sample sizes. They nonetheless are lacking in redundancy, interactivity, stakes or any centered perspective such that there is no chance of anything resembling intelligence, agency, or plain old intentions arising.