6 ms·

The thing that scares me about the Singularity is that it's such potent kool-aid for tech geeks. I talk to Singularitarians and there's this fire in their eyes

by thunk 16y ago

The thing that scares me about the Singularity is that it's such potent kool-aid for tech geeks. I talk to Singularitarians and there's this fire in their eyes like they've been reborn. They become so certain. They remind me of fundamentalists. If there's one thing more dangerous than a true believer it's a crowd of really smart true believers. I'm afraid, perhaps unnecessarily, of what might be done in the name of "ushering in the singularity". I don't think we can know whether a singularity is possible until it's upon us, which could be never. I'm singularity agnostic. That doesn't mean we don't think hard about it. That doesn't mean we don't guard ourselves against its perceived dangers. It does mean that we keep our heads and stay skeptical and put out the fires that are already burning.

skermes 16y ago
It's not for nothing that the Singularity is referred to as "The Rapture of the Geeks". In all seriousness, though, the parallels between some forms of Rapture philosophy and some forms of Singularity philosophy are pretty strong. Uploading your mind into a machine is very similar to being taken up to heaven.
- randallsquared 16y ago
  Well, as long as you overlook the minor detail that the evidence for computers is better than the evidence for heaven. ;)
  - hugh3 16y ago
    A good line, but still doesn't justify the level of certainty which many of these people have in future events turning out the way they think they will. Back in the 90s, as a callow teenager, I happened to be lurking on a mailing list for transhumanists. One day someone on the list, well known to many of the others, died. "Fortunately", as the announcement to the list said, "his body was cryogenically frozen, so we'll all see him again after the singularity!" This was followed by a bunch of posts expressing confidence that this would indeed happen. It was roughly at this point that I decided that these people may be somewhat disconnected from reality, and living in a fantasyland of what they'd like to be true -- while I admit it's possible that this guy will one day be revived, there's no goddamn reason to have any confidence in it happening.
    - randallsquared 16y ago
      I don't disagree. In fact, there's a whole branch of transhumanists (SIAI and fellow travelers) who may be even more pessimistic than you are, but don't view that as a reason to give up trying for a beneficial Singularity. Me, I'm concentrating on enjoying the ride, and if it turns out that humanity (including existing people) isn't wiped out by mid-century, that'll be a most pleasant surprise.
JoshuaZ 16y ago
I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity." The worst plausible scenario is that we put money into research avenues that don't pan out. But that happens all the time anyways. The real worst case scenario is that someone gets too afraid of AI and goes all Sarah Connor. But that's not a likely scenario especially because most of the people who take the Singularity idea seriously are people who are generally optimistic about the long-term trends.
- thunk 16y ago
  There's nothing I'd put past a true believer of any stripe.
- randallsquared 16y ago
  I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity." How about firing up a potentially runaway AI without a stable goal system? Releasing a virus to "improve" people in the releasers' preferred direction? The likelihood of these seems minor (and for the first, might not even been possible), but the consequences are so grave that they bear some thought and worry anyway.
asdflkj 16y ago
I'm afraid too--that a lot of smart people are flushing their time down the toilet. It seems like all things that actually bring us closer to singularity (if the word means anything at all) are being done by people who've never heard of it, or might as well have never heard of it.
- pavs 16y ago
  I doubt you will find anyone on the tech/scientist community who is making major innovations and discoveries never heard of the idea of technological singularities. The idea has been around or a while.
strebler 16y ago
You just perfectly summarized my feelings on the matter; that "Singularity mentality" is kind of creepy.

by thunk 16y ago

skermes 16y ago

It's not for nothing that the Singularity is referred to as "The Rapture of the Geeks". In all seriousness, though, the parallels between some forms of Rapture philosophy and some forms of Singularity philosophy are pretty strong. Uploading your mind into a machine is very similar to being taken up to heaven.

randallsquared 16y ago
Well, as long as you overlook the minor detail that the evidence for computers is better than the evidence for heaven. ;)
- hugh3 16y ago
  A good line, but still doesn't justify the level of certainty which many of these people have in future events turning out the way they think they will. Back in the 90s, as a callow teenager, I happened to be lurking on a mailing list for transhumanists. One day someone on the list, well known to many of the others, died. "Fortunately", as the announcement to the list said, "his body was cryogenically frozen, so we'll all see him again after the singularity!" This was followed by a bunch of posts expressing confidence that this would indeed happen. It was roughly at this point that I decided that these people may be somewhat disconnected from reality, and living in a fantasyland of what they'd like to be true -- while I admit it's possible that this guy will one day be revived, there's no goddamn reason to have any confidence in it happening.
  - randallsquared 16y ago
    I don't disagree. In fact, there's a whole branch of transhumanists (SIAI and fellow travelers) who may be even more pessimistic than you are, but don't view that as a reason to give up trying for a beneficial Singularity. Me, I'm concentrating on enjoying the ride, and if it turns out that humanity (including existing people) isn't wiped out by mid-century, that'll be a most pleasant surprise.

JoshuaZ 16y ago

I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity." The worst plausible scenario is that we put money into research avenues that don't pan out. But that happens all the time anyways. The real worst case scenario is that someone gets too afraid of AI and goes all Sarah Connor. But that's not a likely scenario especially because most of the people who take the Singularity idea seriously are people who are generally optimistic about the long-term trends.

thunk 16y ago
There's nothing I'd put past a true believer of any stripe.
randallsquared 16y ago
I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity." How about firing up a potentially runaway AI without a stable goal system? Releasing a virus to "improve" people in the releasers' preferred direction? The likelihood of these seems minor (and for the first, might not even been possible), but the consequences are so grave that they bear some thought and worry anyway.

asdflkj 16y ago

I'm afraid too--that a lot of smart people are flushing their time down the toilet. It seems like all things that actually bring us closer to singularity (if the word means anything at all) are being done by people who've never heard of it, or might as well have never heard of it.

pavs 16y ago
I doubt you will find anyone on the tech/scientist community who is making major innovations and discoveries never heard of the idea of technological singularities. The idea has been around or a while.

strebler 16y ago

You just perfectly summarized my feelings on the matter; that "Singularity mentality" is kind of creepy.