3 ms·

There are so many criticism/debunk of the Singularity theory that there's not even a debate anymore. I would say that debating the scientific likelihood of a s

There are so many criticism/debunk of the Singularity theory that there's not even a debate anymore. I would say that debating the scientific likelihood of a singularity is sterile because they don't provide any argument on that plane. The debate is mostly philosophical and/or theological nowadays. One good paper I like on the subject is this one: https://jods.mitpress.mit.edu/pub/resisting-reduction https://jods.mitpress.mit.edu/pub/resisting-reduction but quickly googling "singularity criticism/debunk" will bring you so many sources.

pas 8y ago
I've suffered through this ever waiting for it to engage with any of the arguments from MIRI, Bostrom and co. yet it only managed to paint a few strange windmills for itself to charge at. So it talks about Singularists, and how misguided they are, and how AI is a misnomer and it should be called EI and namedrops cybernetics and systems and reminisces about how we lost our way. It is a bad manifesto, a boring and toothless essay. It has a good point, but alas I forget it as I tried to keep reading. Anyway, let's just take this sentence: "We can measure the ability for systems to adapt creatively, as well as their resilience and their ability to use resources in an interesting way." - This is so broad and universal, that it's either meaningless or false. And there is absolutely no argument supporting this. I highlighted this, because this is what directly contradicts the alarmist thinking. If we were able to measure creativity and resilience in general, we could train AIs to get a higher creativity score, furthermore, we could then control them. And it's also interesting that this claim goes counter to a claim a bit earlier about how unknowable and messy things are going to be. It also somehow picks corporations as the perfect model for a superintelligence, which is convenient, but by doing so sidesteps all the real arguments about how a self-perfecting machine superintelligence is not bound by slow components. And somehow also ignores the reality of Samsung, AIG, MicroSoft, Oracle, Shell, BP and how all got away with almost everything. (And other companies that are quite successful, despite competition. And how much we have to work to keep them sort of aligned to our laws and goals.) The best argument against a hard take off is that it's hard to imagine that so many S-curves can be combed through in so little time with the real world resource constraints. However Yudkowsky did an analysis of that: https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf and sure, it's just a step toward more questions, more hard to imagine things, but not something that should be dismissed just because our mind throws up its hands and says "i don't see how, it's very complex, so unlikely, let's go shopping". And this essay is exactly that. It goes on and on about those blind Singularists, and completley misses the point.
- pdimitar 8y ago
  In general, I am in agreement with you. The whole thing started resembling religious wars in the last years and it's quite hilarious and sad to observe. This part however I can't stand behind: > Anyway, let's just take this sentence: "We can measure the ability for systems to adapt creatively, as well as their resilience and their ability to use resources in an interesting way." - This is so broad and universal, that it's either meaningless or false. And there is absolutely no argument supporting this. I highlighted this, because this is what directly contradicts the alarmist thinking. If we were able to measure creativity and resilience in general, we could train AIs to get a higher creativity score, furthermore, we could then control them. It's true that such generalist statements basically mean nothing. But your rebuttal doesn't take into account chaos theory -- where there's generally accepted that most living systems live on the brink of chaos yet are very stable and manage to swing back even after big interferences. Not sure what a "living system" is, don't ask. :D I do agree with you that your highlight contradicts the alarmists though. Not everything swings out of control by the gentlest of touches. In fact, most of the universe doesn't seem to be that way. There's always a lot of critical mass that must be accumulated before a cataclysm-like event occurs. --- All of the above said, I don't think it's serious or scientific to discard alarmists simply because the guys/girls with the most PR do ridiculuous or non-scientific statements. Behind them are probably thousands of people who are more systemic and have better arguments but aren't interviewed by mainstream media. Where do I stand in the spectrum of this? The so-called singularity is possible. BUT, we are a very long way from it. We're going to be clawing our way to it, inch by inch, for centuries, if not millenia. That's what I think is most likely. And by the time it occurs, IMO we will be living in a cyberpunk-like future -- very well articulated in the "Ghost in the Shell" anime movies and series by the way -- where the line between a man and machine will have already be blurred quite severely.

pas 8y ago

I've suffered through this ever waiting for it to engage with any of the arguments from MIRI, Bostrom and co. yet it only managed to paint a few strange windmills for itself to charge at. So it talks about Singularists, and how misguided they are, and how AI is a misnomer and it should be called EI and namedrops cybernetics and systems and reminisces about how we lost our way. It is a bad manifesto, a boring and toothless essay. It has a good point, but alas I forget it as I tried to keep reading. Anyway, let's just take this sentence: "We can measure the ability for systems to adapt creatively, as well as their resilience and their ability to use resources in an interesting way." - This is so broad and universal, that it's either meaningless or false. And there is absolutely no argument supporting this. I highlighted this, because this is what directly contradicts the alarmist thinking. If we were able to measure creativity and resilience in general, we could train AIs to get a higher creativity score, furthermore, we could then control them. And it's also interesting that this claim goes counter to a claim a bit earlier about how unknowable and messy things are going to be. It also somehow picks corporations as the perfect model for a superintelligence, which is convenient, but by doing so sidesteps all the real arguments about how a self-perfecting machine superintelligence is not bound by slow components. And somehow also ignores the reality of Samsung, AIG, MicroSoft, Oracle, Shell, BP and how all got away with almost everything. (And other companies that are quite successful, despite competition. And how much we have to work to keep them sort of aligned to our laws and goals.) The best argument against a hard take off is that it's hard to imagine that so many S-curves can be combed through in so little time with the real world resource constraints. However Yudkowsky did an analysis of that: https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf and sure, it's just a step toward more questions, more hard to imagine things, but not something that should be dismissed just because our mind throws up its hands and says "i don't see how, it's very complex, so unlikely, let's go shopping". And this essay is exactly that. It goes on and on about those blind Singularists, and completley misses the point.

pdimitar 8y ago
In general, I am in agreement with you. The whole thing started resembling religious wars in the last years and it's quite hilarious and sad to observe. This part however I can't stand behind: > Anyway, let's just take this sentence: "We can measure the ability for systems to adapt creatively, as well as their resilience and their ability to use resources in an interesting way." - This is so broad and universal, that it's either meaningless or false. And there is absolutely no argument supporting this. I highlighted this, because this is what directly contradicts the alarmist thinking. If we were able to measure creativity and resilience in general, we could train AIs to get a higher creativity score, furthermore, we could then control them. It's true that such generalist statements basically mean nothing. But your rebuttal doesn't take into account chaos theory -- where there's generally accepted that most living systems live on the brink of chaos yet are very stable and manage to swing back even after big interferences. Not sure what a "living system" is, don't ask. :D I do agree with you that your highlight contradicts the alarmists though. Not everything swings out of control by the gentlest of touches. In fact, most of the universe doesn't seem to be that way. There's always a lot of critical mass that must be accumulated before a cataclysm-like event occurs. --- All of the above said, I don't think it's serious or scientific to discard alarmists simply because the guys/girls with the most PR do ridiculuous or non-scientific statements. Behind them are probably thousands of people who are more systemic and have better arguments but aren't interviewed by mainstream media. Where do I stand in the spectrum of this? The so-called singularity is possible. BUT, we are a very long way from it. We're going to be clawing our way to it, inch by inch, for centuries, if not millenia. That's what I think is most likely. And by the time it occurs, IMO we will be living in a cyberpunk-like future -- very well articulated in the "Ghost in the Shell" anime movies and series by the way -- where the line between a man and machine will have already be blurred quite severely.