3 ms·

This whole thread is very stereotypically EA to me -- and why I see it as a sham. Starts off at the top saying how it's all about donating to reasonable chariti

This whole thread is very stereotypically EA to me -- and why I see it as a sham. Starts off at the top saying how it's all about donating to reasonable charities, but soon devolved into "what about muh AGI". I am literally doing my Ph.D. in robot learning and I can tell you that every single "longtermist" effective altruist I have talked to has been more interested in getting high on their own stash of Sci Fi fantasies rather than interacting with the actual practitioners of the field and understanding what is happening, what are the very real limitations of it, and why their assumptions range from willfully ignorant to often simply maliciously stupid. It's almost as if the wealthy, the privileged, and the powerful are interested in indulgences while doing things that tickles their fantasies, and as the invisible hand of the market works, there is a class of "philosophers" who are willing to sell it to them.

naasking 4y ago
> rather than interacting with the actual practitioners of the field and understanding what is happening, what are the very real limitations of it, and why their assumptions range from willfully ignorant to often simply maliciously stupid. On the other hand, I've found that "practitioners in the field" often consistently overestimate how complex and different the human mind is from our computers. Take the recent "LaMDA is sentient" debate. Machine learning researchers consistently and vehemently denied that LaMDA is sentient. They no doubt have a deep and subtle understanding of how LaMDA works, but apparently they didn't realize that we have no mechanistic understanding of what sentience actually is or how it works, so they have absolutely no basis for such a conclusion. So on the one hand we have these AGI doomsayers that, per your evaluation, don't understand machine learning subtleties, and on the other hand we have some machine learning researchers skeptical of AGI who consistently overestimate human specialness. At this time the error bars I would assign to each side are wide enough that there's lots of overlap, but given the track record of human specialness falling to algorithmic progress, I'd say some investment in AGI safety is prudent.
- danaris 4y ago
  You don't need to have a deep and complex understanding of human sentience to be able to put some lower bounds on what it means to be sentient. I don't know about sentience, as such; however, I believe there are at least two bare-bones prerequisites to an AI being conscious as we understand the term: 1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. 2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives. Now, these are probably not sufficient for an AI to be conscious by our understanding of consciousness. But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
  - naasking 4y ago
    > Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. Why can't it be conscious merely while the computation is running? You aren't constantly on, you're ostensibly only conscious 16-20 hours every day, while you're not sleeping. > Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives. I'm not sure what learning has to do with consciousness, and I'm absolutely certain you don't either. I frankly find it hard to believe that Zen Buddhists are not conscious during their meditations, despite having empty minds and not "constantly updating their mental models of the world-in-effect". > But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience. See, this is what I'm talking about. That's a fine personal belief, but that belief does not rise to the level of expert testimony of the facts even if you are a machine learning expert. People have made all sorts of assumptions about the human mind that they have never questioned but take as obvious fact, and then pontificate that AI is clearly not doing this, therefore AI cannot share such and such properties with humans. But the fact is, most of those assumptions are just wrong. Without a mechanistic model of how these systems work in the brain we are not going to make a valid comparison against what AI is doing. I agree in general that we can set some bounds, but those bounds are neither as firm as, nor are they placed where most people think.

naasking 4y ago

> rather than interacting with the actual practitioners of the field and understanding what is happening, what are the very real limitations of it, and why their assumptions range from willfully ignorant to often simply maliciously stupid. On the other hand, I've found that "practitioners in the field" often consistently overestimate how complex and different the human mind is from our computers. Take the recent "LaMDA is sentient" debate. Machine learning researchers consistently and vehemently denied that LaMDA is sentient. They no doubt have a deep and subtle understanding of how LaMDA works, but apparently they didn't realize that we have no mechanistic understanding of what sentience actually is or how it works, so they have absolutely no basis for such a conclusion. So on the one hand we have these AGI doomsayers that, per your evaluation, don't understand machine learning subtleties, and on the other hand we have some machine learning researchers skeptical of AGI who consistently overestimate human specialness. At this time the error bars I would assign to each side are wide enough that there's lots of overlap, but given the track record of human specialness falling to algorithmic progress, I'd say some investment in AGI safety is prudent.

danaris 4y ago
You don't need to have a deep and complex understanding of human sentience to be able to put some lower bounds on what it means to be sentient. I don't know about sentience, as such; however, I believe there are at least two bare-bones prerequisites to an AI being conscious as we understand the term: 1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. 2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives. Now, these are probably not sufficient for an AI to be conscious by our understanding of consciousness. But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
- naasking 4y ago
  > Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. Why can't it be conscious merely while the computation is running? You aren't constantly on, you're ostensibly only conscious 16-20 hours every day, while you're not sleeping. > Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives. I'm not sure what learning has to do with consciousness, and I'm absolutely certain you don't either. I frankly find it hard to believe that Zen Buddhists are not conscious during their meditations, despite having empty minds and not "constantly updating their mental models of the world-in-effect". > But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience. See, this is what I'm talking about. That's a fine personal belief, but that belief does not rise to the level of expert testimony of the facts even if you are a machine learning expert. People have made all sorts of assumptions about the human mind that they have never questioned but take as obvious fact, and then pontificate that AI is clearly not doing this, therefore AI cannot share such and such properties with humans. But the fact is, most of those assumptions are just wrong. Without a mechanistic model of how these systems work in the brain we are not going to make a valid comparison against what AI is doing. I agree in general that we can set some bounds, but those bounds are neither as firm as, nor are they placed where most people think.