4 ms·

"There has been a domineering subculture—that's been the most wealthy, prolific, and influential subculture in the technical world—that for a long time has not

by VLM 11y ago

"There has been a domineering subculture—that's been the most wealthy, prolific, and influential subculture in the technical world—that for a long time has not only promoted the idea that there's an equivalence between algorithms and life, and certain algorithms and people, but a historical determinism that we're inevitably making computers that will be smarter and better than us and will take over from us." On a different discussion site I recently pointed out there is a similar religious belief or religious dogma in the idea of the self driving car, that went over like a lead balloon. People acting in a religious manner and expressing religious belief don't like to have that pointed out to them. Probably vestigial monotheism, if they act religiously at their traditional church on Sundays they don't like it pointed out that they worship a different altar online, or the atheists get really wound up when they are called out for joining the atheist movement but acting as deacons of the church of the self driving car of the future. Anyway just saying its a line of reasoning, that while true, can only lead to unpleasantness. Like discussion of scientific racial / gender differences or discussions about IQ, all you're gonna get is social signalling screaming "see no evil".

randallsquared 11y ago
While I don't agree with Lanier, being an AI skeptic isn't quite the same as saying that self driving cars can't happen: we don't yet have a good understanding of either consciousness or tractable general optimization algorithms, so it's still possible that there's something that will make it impossible. Unlikely, perhaps, but possible. Also, your sarcasm about the self driving car might have been too subtle. ;)
- ghaff 11y ago
  What you see with respect to self-driving cars IMO is a lot of people who SO want them to exist that they assign timelines that are so optimistic they'd be rolling on the floor laughing if applied to projects they had direct familiarity with. Of course, it's also fueled by a lot of hype and hand-wavy statements about exponential growth and the like. Like many other aspects of AI, I'm not sure just about anyone thinks we need to have strong AI to make self-driving cars work under the vast majority of conditions. We "just" need to create automation that can handle a reasonable number of corner cases without flipping out or grinding to a halt. But that's a debate about engineering timelines not something more philosophical.
  - randallsquared 11y ago
    I'm a little confused. We have self-driving cars. Of course there are still corner cases and bugs and regulatory issues, but cars have driven for fair distances on existing roads without a human driving. I had assumed, therefore, that the point @VLM was making was that, like self-driving cars, we already have AI by many of the standards of earlier AI researchers (the "when it works, it's not called AI any longer" argument).
    - ghaff 11y ago
      I was reading "self-driving cars" as "Level 4 autonomy" under general conditions. Which IMO is many decades away. But we're in violent agreement about the "when it works, it's not called AI any longer" argument.
AndrewKemendo 11y ago
vestigial monotheism That's a fantastic metaphor. I'm curious though how someone could act in a religious manner about a self-driving car. To the original point though, I am very much in the camp of "historical determinism that we're inevitably making computers that will be smarter and better than us" and here is why: I believe that intelligence (and "consciousness" if you want to go down that rabbit hole) is completely material and as such it is possible that we will eventually understand the mechanics of them. If we can understand those, then history would indicate (Historical fallacy) that we can replace or replicate those mechanics with more durable systems or materials than the fairly fragile.
- TheOtherHobbes 11y ago
  That's a religious position. We have absolutely no idea what consciousness is or how it works. More than that, we have don't even have ad hoc good models for motivation and personality, never mind useful formal and explicit models. I think Lanier is absolutely right, and I think there's a strong quasi-theological element running through all of AI and programming. Programming is a lot like making spells and incantations. If you get them just so you get the outcome you want. But you have to speak the language of the system to make that possible. And you have to be very careful about unintended consequences. There's something very medieval about this - both in the sense of the pure scholasticism of academic CS, but also in the practical sense of knowing how to formulate the correct "prayers" to make useful things happen. Lanier seems to be pointing out that CS is still haunted and influenced by these religious metaphors, and that AI is the most visible example of that. I think he's right - and more, I think that real AI, in the sense of autonomous personalities, won't be possible until that's no longer true.
  - AndrewKemendo 11y ago
    How is it a religious position? A religious position would be thinking that some unknowable process would deliver some kind of techno-rapture. So because we don't know how something works now, we will never know? C'mon. Programming is a lot like making spells and incantations. No, it's not. Maybe for laymen it looks like that, but for us developers it is definitely not the case. Granted, sometimes something works and you don't realize why, but that just means you have a big debug problem on your hands - not that it's unknowable. Usually only happens when you port something or copy code over from one system to the next. Not sure how you come to those conclusions.
golemotron 11y ago
> Anyway just saying its a line of reasoning, that while true, can only lead to unpleasantness. Like discussion of scientific racial / gender differences or discussions about IQ, all you're gonna get is social signalling screaming "see no evil". I'm waiting for it to fall apart. The amount of information that you have to willingly ignore these days to participate in the ''progress solves all ills'' narrative is huge. On vestigial monotheism read Straw Dogs by John Gray if you haven't.
Jach 11y ago
https://web.archive.org/web/20140425185111/http://www.acceleratingfuture.com/steven/?p=21 https://web.archive.org/web/20140425185111/http://www.accele...
pron 11y ago
I fully agree with the religious aspect of it (religion doesn't have to be theistic at all, but the particular beliefs in an after-life/resurrection/omniscient-omnipotent being do resemble the theistic, and indeed, monotheistic religions), but I think there's another aspect to the "AI threat" belief that is more than just "vestigial monotheism". I think this is (in part) a power fantasy. The believers in the AI threat normally perceive themselves to be different from other people due to their (self-perceived or real) higher intelligence. They therefore describe that property (that they believe makes them special) as being particularly dangerous, and therefore powerful, because they want to believe that intelligence imbues its possessor with power and a lot of it. Sadly, superbugs, meteorites, H bombs, climate change, human-rights violations, inequality and other threats don't require or possess much intelligence, but an unstoppable AI is the purest manifestation of intelligence-as-power. In the human world, intelligence is not very correlated with power, except very roughly. The most powerful people are generally more intelligent than average, but the correlation doesn't extend far beyond that. If anything, charm, confidence and courage seem to imbue their bearers with much more power than raw intelligence. Which is why I fear (only partly jokingly) a super-charming machine than a super-intelligent one. A super-charming machine is far more likely to bend humanity to its will -- and therefore pose a greater danger -- even if what it wants may be truly idiotic; a super-intelligent machine would be far more limited in its effects...
- VLM 11y ago
  "A super-charming machine" A television, or a tablet with facebook and youtube apps? Its interesting to think of mass propaganda a la Bernays as an example of "super charming machine". In the old days the cogs and wheels in the marketing business are/were humans, just like in the old days "math calculator" was a job description but it got automated and accelerated into computers. Populist political machine might be another example.
- tome 11y ago
  > unstoppable AI is the purest manifestation of intelligence-as-power Brilliant observation.
api 11y ago
Religion and ideological herd behaviors are a core part of human nature. You can't expect these to magically vanish just because people have abandoned belief in supernatural deities and magic. Those are just one form of it. Religion is a social phenomenon and cognitive style that is independent from the subject matter. That being said: the techno-critics also display these behaviors as do the new agers, "naturalists," and others with which they tend to travel. Head over to one of those circles or boards and start arguing the futurist line and see what it gets you. Both sides have valid points.
language 11y ago
With regard to "vestigial monotheism": I think what you're describing is just a property of discourses in general, in that they're all built around bodies of knowledge which have fixed points [assumptions/truths/whatever], and that participants act in such a way as to keep those things stable. Definitely seen things like this before, very poignant. Not to knock on Kurzweil and friends too hard [lots of interesting stuff in both spaces obviously], but I've had this vibe from talking with self-described Singularitarians as well (there's some overlap between those circles).
ArkyBeagle 11y ago
It is kind of religion - sort of classical American Yankee machine-worship. Even the Puritans/Pilgrims thought that organization - essentially a technology - could save them from the swamp gases of British life. And I can't say it didn't work, but I'm probably a fanatic at this religion anyway. I think Mark Twain more or less says this outright in "Connecticut Yankee." Speaking of Twain, he grievously injured himself financially chasing the dream of a typesetting machine far ahead of its time. If it's religion, it's religion that (mostly) works. <insert Panglossian diatribe here>

by VLM 11y ago

randallsquared 11y ago

While I don't agree with Lanier, being an AI skeptic isn't quite the same as saying that self driving cars can't happen: we don't yet have a good understanding of either consciousness or tractable general optimization algorithms, so it's still possible that there's something that will make it impossible. Unlikely, perhaps, but possible. Also, your sarcasm about the self driving car might have been too subtle. ;)

ghaff 11y ago
What you see with respect to self-driving cars IMO is a lot of people who SO want them to exist that they assign timelines that are so optimistic they'd be rolling on the floor laughing if applied to projects they had direct familiarity with. Of course, it's also fueled by a lot of hype and hand-wavy statements about exponential growth and the like. Like many other aspects of AI, I'm not sure just about anyone thinks we need to have strong AI to make self-driving cars work under the vast majority of conditions. We "just" need to create automation that can handle a reasonable number of corner cases without flipping out or grinding to a halt. But that's a debate about engineering timelines not something more philosophical.
- randallsquared 11y ago
  I'm a little confused. We have self-driving cars. Of course there are still corner cases and bugs and regulatory issues, but cars have driven for fair distances on existing roads without a human driving. I had assumed, therefore, that the point @VLM was making was that, like self-driving cars, we already have AI by many of the standards of earlier AI researchers (the "when it works, it's not called AI any longer" argument).
  - ghaff 11y ago
    I was reading "self-driving cars" as "Level 4 autonomy" under general conditions. Which IMO is many decades away. But we're in violent agreement about the "when it works, it's not called AI any longer" argument.

AndrewKemendo 11y ago

vestigial monotheism That's a fantastic metaphor. I'm curious though how someone could act in a religious manner about a self-driving car. To the original point though, I am very much in the camp of "historical determinism that we're inevitably making computers that will be smarter and better than us" and here is why: I believe that intelligence (and "consciousness" if you want to go down that rabbit hole) is completely material and as such it is possible that we will eventually understand the mechanics of them. If we can understand those, then history would indicate (Historical fallacy) that we can replace or replicate those mechanics with more durable systems or materials than the fairly fragile.

TheOtherHobbes 11y ago
That's a religious position. We have absolutely no idea what consciousness is or how it works. More than that, we have don't even have ad hoc good models for motivation and personality, never mind useful formal and explicit models. I think Lanier is absolutely right, and I think there's a strong quasi-theological element running through all of AI and programming. Programming is a lot like making spells and incantations. If you get them just so you get the outcome you want. But you have to speak the language of the system to make that possible. And you have to be very careful about unintended consequences. There's something very medieval about this - both in the sense of the pure scholasticism of academic CS, but also in the practical sense of knowing how to formulate the correct "prayers" to make useful things happen. Lanier seems to be pointing out that CS is still haunted and influenced by these religious metaphors, and that AI is the most visible example of that. I think he's right - and more, I think that real AI, in the sense of autonomous personalities, won't be possible until that's no longer true.
- AndrewKemendo 11y ago
  How is it a religious position? A religious position would be thinking that some unknowable process would deliver some kind of techno-rapture. So because we don't know how something works now, we will never know? C'mon. Programming is a lot like making spells and incantations. No, it's not. Maybe for laymen it looks like that, but for us developers it is definitely not the case. Granted, sometimes something works and you don't realize why, but that just means you have a big debug problem on your hands - not that it's unknowable. Usually only happens when you port something or copy code over from one system to the next. Not sure how you come to those conclusions.

golemotron 11y ago

> Anyway just saying its a line of reasoning, that while true, can only lead to unpleasantness. Like discussion of scientific racial / gender differences or discussions about IQ, all you're gonna get is social signalling screaming "see no evil". I'm waiting for it to fall apart. The amount of information that you have to willingly ignore these days to participate in the ''progress solves all ills'' narrative is huge. On vestigial monotheism read Straw Dogs by John Gray if you haven't.

Jach 11y ago

https://web.archive.org/web/20140425185111/http://www.acceleratingfuture.com/steven/?p=21 https://web.archive.org/web/20140425185111/http://www.accele...

pron 11y ago

I fully agree with the religious aspect of it (religion doesn't have to be theistic at all, but the particular beliefs in an after-life/resurrection/omniscient-omnipotent being do resemble the theistic, and indeed, monotheistic religions), but I think there's another aspect to the "AI threat" belief that is more than just "vestigial monotheism". I think this is (in part) a power fantasy. The believers in the AI threat normally perceive themselves to be different from other people due to their (self-perceived or real) higher intelligence. They therefore describe that property (that they believe makes them special) as being particularly dangerous, and therefore powerful, because they want to believe that intelligence imbues its possessor with power and a lot of it. Sadly, superbugs, meteorites, H bombs, climate change, human-rights violations, inequality and other threats don't require or possess much intelligence, but an unstoppable AI is the purest manifestation of intelligence-as-power. In the human world, intelligence is not very correlated with power, except very roughly. The most powerful people are generally more intelligent than average, but the correlation doesn't extend far beyond that. If anything, charm, confidence and courage seem to imbue their bearers with much more power than raw intelligence. Which is why I fear (only partly jokingly) a super-charming machine than a super-intelligent one. A super-charming machine is far more likely to bend humanity to its will -- and therefore pose a greater danger -- even if what it wants may be truly idiotic; a super-intelligent machine would be far more limited in its effects...

VLM 11y ago
"A super-charming machine" A television, or a tablet with facebook and youtube apps? Its interesting to think of mass propaganda a la Bernays as an example of "super charming machine". In the old days the cogs and wheels in the marketing business are/were humans, just like in the old days "math calculator" was a job description but it got automated and accelerated into computers. Populist political machine might be another example.
tome 11y ago
> unstoppable AI is the purest manifestation of intelligence-as-power Brilliant observation.

api 11y ago

Religion and ideological herd behaviors are a core part of human nature. You can't expect these to magically vanish just because people have abandoned belief in supernatural deities and magic. Those are just one form of it. Religion is a social phenomenon and cognitive style that is independent from the subject matter. That being said: the techno-critics also display these behaviors as do the new agers, "naturalists," and others with which they tend to travel. Head over to one of those circles or boards and start arguing the futurist line and see what it gets you. Both sides have valid points.

language 11y ago

With regard to "vestigial monotheism": I think what you're describing is just a property of discourses in general, in that they're all built around bodies of knowledge which have fixed points [assumptions/truths/whatever], and that participants act in such a way as to keep those things stable. Definitely seen things like this before, very poignant. Not to knock on Kurzweil and friends too hard [lots of interesting stuff in both spaces obviously], but I've had this vibe from talking with self-described Singularitarians as well (there's some overlap between those circles).

ArkyBeagle 11y ago

It is kind of religion - sort of classical American Yankee machine-worship. Even the Puritans/Pilgrims thought that organization - essentially a technology - could save them from the swamp gases of British life. And I can't say it didn't work, but I'm probably a fanatic at this religion anyway. I think Mark Twain more or less says this outright in "Connecticut Yankee." Speaking of Twain, he grievously injured himself financially chasing the dream of a typesetting machine far ahead of its time. If it's religion, it's religion that (mostly) works. <insert Panglossian diatribe here>