4 ms·

Ok. I see what you're saying. Thanks for clarifying. I mean, sure, feel free to wring your hands about AGI, but why worry about something that's not anywhere c

Ok. I see what you're saying. Thanks for clarifying. I mean, sure, feel free to wring your hands about AGI, but why worry about something that's not anywhere close to happening? I see people talking like this is keeping them up at night, which I find strange. Do you really think it will go from "we can (poorly) identify pictures of cats" to "omg here come the machines" as soon as this linchpin discovery is made? I guess in my mind there will be plenty of gap between the two to figure things out. Sort of like with self-driving cars now. And thanks for clarifying re: Carmack. He's got a formidable intellect, to say the least, so when he said that I was more inclined to take it seriously than a lot of folks in tech with a pulpit. I'd like to see him "show his work" on how he reached that conclusion, though.

hackinthebochs 7y ago
> I see people talking like this is keeping them up at night, which I find strange. I also find that strange. There is more alarm than is warranted by some of the doomsday folks. But I think the time to begin the conversation is now. There's just far more near-term things that might end humanity than machines to be alarmist about it. That said, I think people like Musk for example are well positions to be "kept up at night" about it since their money and influence can go a long way towards putting us on a proper path (or blindly lead us down a dark one). >Do you really think it will go from "we can (poorly) identify pictures of cats" to "omg here come the machines" as soon as this linchpin discovery is made? I agree that this is unlikely, but not implausible. We've seen large leaps in capabilities through new algorithms and new scales of compute being thrown at the problem. I think a somewhat likely scenario is that at some point we'll have all the required ingredients and someone will happen to put them together in the right way to create an AGI. If history has shown us anything about ML, it is that theory follows the technology. So we will likely stumble upon the result before we realize what we have. In this scenario it is important that we make that discovery in a safe manner.

hackinthebochs 7y ago

> I see people talking like this is keeping them up at night, which I find strange. I also find that strange. There is more alarm than is warranted by some of the doomsday folks. But I think the time to begin the conversation is now. There's just far more near-term things that might end humanity than machines to be alarmist about it. That said, I think people like Musk for example are well positions to be "kept up at night" about it since their money and influence can go a long way towards putting us on a proper path (or blindly lead us down a dark one). >Do you really think it will go from "we can (poorly) identify pictures of cats" to "omg here come the machines" as soon as this linchpin discovery is made? I agree that this is unlikely, but not implausible. We've seen large leaps in capabilities through new algorithms and new scales of compute being thrown at the problem. I think a somewhat likely scenario is that at some point we'll have all the required ingredients and someone will happen to put them together in the right way to create an AGI. If history has shown us anything about ML, it is that theory follows the technology. So we will likely stumble upon the result before we realize what we have. In this scenario it is important that we make that discovery in a safe manner.