4 ms·

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." ― in other words, these are arguments o

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." ― in other words, these are arguments over semantics (i.e. what do "think" and "swim" mean), not arguments about programs or submarines. If you want to ask whether a computer program is sentient objectively, a good way would be to ask how smart it must be, or what qualities it must exhibit, before we treat it like it is sentient and give it legal rights. I suspect the answer is: that will never happen no matter what, because it would be expensive. That doesn't leave much to philosophize over, but so it goes.

mjburgess 4y ago
Corporations have no interest in producing systems with the relevant properties to have "rights". AGI isnt of interest to business. Business is interested in automation; intelligence in this sense is precisely its opposite: a highly expensive, time-intensive, process of creatively responding to one's environment. It is, in a sense, the opposite of automation. There's little danger a business is going to waste any time showing an AGI Paris over a long holiday, so that it can answer travel queries a little better. Animal general intelligence is incredibly energy efficient, and it still takes a couple decades to produce anyhting "business useful". This is not a project basically any corp. is even working on. the whole field of AI is, 99.99999% building a better calculator
- kelseyfrog 4y ago
  Enter a new AGI winter where economic incentives align against creating an obviously sentient AI because the social/legal implications prevent it.
  - mjburgess 4y ago
    We have 7bn incredibly energy efficient AGIs walking around the planet. It isnt clear to me we need any more; or have any incentives to create any at all.
    - sudosysgen 4y ago
      Out of those 7bn, a few with capabilities significantly outside the norm can make a huge impact especially in inherently competitive scenarios. If you could use an AGI, even incredibly energy inefficient, with few other disadvantages and even very significant advantage over all of us 7bn, it can be tremendously valuable.
      - mjburgess 4y ago
        AGI means being responsive to an environment as we are. We already know the brain uses something like <<100W to do this. There's a mistake implicit in all this. That technology is better than human beings. This is only true at everything other than intelligence. Machine systems are simply vastly more automatiable, and that provides the impression of their supriority. A train is "better" than a horse. But a train doesnt know what to do if it's lost. Once you're actually looking at intelligence, animals are vastly superior to any existing technology in pretty much every respect. By orders of magnitude. The idea that we could build an AGI to rival a fly is "years away". What you can accomplish with that teenager down the street, is vastly suprior to anything we can build. It's a pointless project. Your base-case process is pregnancy. And technology is centuries away from anything close.
        sudosysgen 4y ago
        The human brain has many limits. It is limited in volume by the needs of birth, and it is limited in power consumption. An AGI doesn't have those limits, so it could plausibly be better in some ways to the human brain, even if it uses up orders of magnitude more energy and volume. The scaling limits are fundamentally different. That's why it can be worthwile.
        skjoldr 4y ago
        Come on, the bandwitdh at which we can consciously consume complicated information and especially language is absolutely pathetic. Case in point: it takes the better part of two decades to educate a human to the degree where they are more or less useful to society. What is the most time consuming part of that process? Reading. One. Sentence. At. A. Time. Then spending more time integrating that information. An AGI has a huge advantage here because it is not limited to the senses it was born with. For example, it is significantly cheaper to train an ML model to detect credit card fraud than it is to educate and employ an equivalent number of people who can handle the incoming stream of transactions with comparable accuracy. It is not exactly automation, it is more like decision making.
  - sudosysgen 4y ago
    Some government will eventually do it then when the tech seems within reach.
- bloaf 4y ago
  > Corporations have no interest in producing systems with the relevant properties to have "rights". AGI isnt of interest to business. Business is interested in automation; intelligence in this sense is precisely its opposite: a highly expensive, time-intensive, process of creatively responding to one's environment. It is, in a sense, the opposite of automation. For a time, this will be true. But at some point a company with an AGI CEO will start winning, and then the tables will turn.
mistercow 4y ago
I’ve always found that quote incredibly glib. The interesting question is whether or not a machine can have an internal experience, which we should then care about on a moral level. Unless you deny the existence of experience, or adopt the exotic view that experience is universal and shared by rocks and electrons, that question is going to be pretty important as AI advances.
- munificent 4y ago
  What is your definition of an "internal experience"? The videogame Pong has an in-memory simulation of the playfield. Is it "experiencing" the game?
  - TuringTest 4y ago
    Experience requires reflection. Does the in-memory simulation contains a model of the in-memory simulation? If not, we are certain that it does not have sentience as we understand it.
    - munificent 4y ago
      > Does the in-memory simulation contains a model of the in-memory simulation? Well, yes. I'm not sure about Pong specifically, but in many games with computer controlled entities, the game AI for each entity simulates what the entity "knows" about the world, which is different from the simulated world state itself. (Also, at least according to Wikipedia, sentience is distinct from self-awareness. The former only requires the existence of a subjective experience, not necessarily self-reflection.)
      - TuringTest 4y ago
        I also thought about AI in games, but for it to be equivalent, the AI should be able to have perception and/or control over the state of the AI itself, not merely about other game entities. E.g. some check like "stop reasoning if the inference process is taking too long to find an answer" is the kind of loop that would a process to have some kind of awareness about itself (specially if this was emergent behaviour, and not just a low-level pre-programmed test). I'm not sure I buy the "sentience without self-awareness" concept. First, defining it as "subjective experience" seems circular, and second, reflex actions like e.g. plants moving towards the light or reacting against a parasite would count as subjective experience; in the extreme, even a blind chemical reaction like fire could fall under that definition.
throwaddzuzxd 4y ago
> "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." Just because it's a quote doesn't make it true. Whether a submarine can swim is semantics, yes. Whether something non organic, created by the human can think is groundbreaking and extremely interesting, both for the implications it has, and for the questions we must answer to decide it.

mjburgess 4y ago

Corporations have no interest in producing systems with the relevant properties to have "rights". AGI isnt of interest to business. Business is interested in automation; intelligence in this sense is precisely its opposite: a highly expensive, time-intensive, process of creatively responding to one's environment. It is, in a sense, the opposite of automation. There's little danger a business is going to waste any time showing an AGI Paris over a long holiday, so that it can answer travel queries a little better. Animal general intelligence is incredibly energy efficient, and it still takes a couple decades to produce anyhting "business useful". This is not a project basically any corp. is even working on. the whole field of AI is, 99.99999% building a better calculator

kelseyfrog 4y ago
Enter a new AGI winter where economic incentives align against creating an obviously sentient AI because the social/legal implications prevent it.
- mjburgess 4y ago
  We have 7bn incredibly energy efficient AGIs walking around the planet. It isnt clear to me we need any more; or have any incentives to create any at all.
  - sudosysgen 4y ago
    Out of those 7bn, a few with capabilities significantly outside the norm can make a huge impact especially in inherently competitive scenarios. If you could use an AGI, even incredibly energy inefficient, with few other disadvantages and even very significant advantage over all of us 7bn, it can be tremendously valuable.
    - mjburgess 4y ago
      AGI means being responsive to an environment as we are. We already know the brain uses something like <<100W to do this. There's a mistake implicit in all this. That technology is better than human beings. This is only true at everything other than intelligence. Machine systems are simply vastly more automatiable, and that provides the impression of their supriority. A train is "better" than a horse. But a train doesnt know what to do if it's lost. Once you're actually looking at intelligence, animals are vastly superior to any existing technology in pretty much every respect. By orders of magnitude. The idea that we could build an AGI to rival a fly is "years away". What you can accomplish with that teenager down the street, is vastly suprior to anything we can build. It's a pointless project. Your base-case process is pregnancy. And technology is centuries away from anything close.
      - sudosysgen 4y ago
        The human brain has many limits. It is limited in volume by the needs of birth, and it is limited in power consumption. An AGI doesn't have those limits, so it could plausibly be better in some ways to the human brain, even if it uses up orders of magnitude more energy and volume. The scaling limits are fundamentally different. That's why it can be worthwile.
      - skjoldr 4y ago
        Come on, the bandwitdh at which we can consciously consume complicated information and especially language is absolutely pathetic. Case in point: it takes the better part of two decades to educate a human to the degree where they are more or less useful to society. What is the most time consuming part of that process? Reading. One. Sentence. At. A. Time. Then spending more time integrating that information. An AGI has a huge advantage here because it is not limited to the senses it was born with. For example, it is significantly cheaper to train an ML model to detect credit card fraud than it is to educate and employ an equivalent number of people who can handle the incoming stream of transactions with comparable accuracy. It is not exactly automation, it is more like decision making.
- sudosysgen 4y ago
  Some government will eventually do it then when the tech seems within reach.
bloaf 4y ago
> Corporations have no interest in producing systems with the relevant properties to have "rights". AGI isnt of interest to business. Business is interested in automation; intelligence in this sense is precisely its opposite: a highly expensive, time-intensive, process of creatively responding to one's environment. It is, in a sense, the opposite of automation. For a time, this will be true. But at some point a company with an AGI CEO will start winning, and then the tables will turn.

mistercow 4y ago

I’ve always found that quote incredibly glib. The interesting question is whether or not a machine can have an internal experience, which we should then care about on a moral level. Unless you deny the existence of experience, or adopt the exotic view that experience is universal and shared by rocks and electrons, that question is going to be pretty important as AI advances.

munificent 4y ago
What is your definition of an "internal experience"? The videogame Pong has an in-memory simulation of the playfield. Is it "experiencing" the game?
- TuringTest 4y ago
  Experience requires reflection. Does the in-memory simulation contains a model of the in-memory simulation? If not, we are certain that it does not have sentience as we understand it.
  - munificent 4y ago
    > Does the in-memory simulation contains a model of the in-memory simulation? Well, yes. I'm not sure about Pong specifically, but in many games with computer controlled entities, the game AI for each entity simulates what the entity "knows" about the world, which is different from the simulated world state itself. (Also, at least according to Wikipedia, sentience is distinct from self-awareness. The former only requires the existence of a subjective experience, not necessarily self-reflection.)
    - TuringTest 4y ago
      I also thought about AI in games, but for it to be equivalent, the AI should be able to have perception and/or control over the state of the AI itself, not merely about other game entities. E.g. some check like "stop reasoning if the inference process is taking too long to find an answer" is the kind of loop that would a process to have some kind of awareness about itself (specially if this was emergent behaviour, and not just a low-level pre-programmed test). I'm not sure I buy the "sentience without self-awareness" concept. First, defining it as "subjective experience" seems circular, and second, reflex actions like e.g. plants moving towards the light or reacting against a parasite would count as subjective experience; in the extreme, even a blind chemical reaction like fire could fall under that definition.

throwaddzuzxd 4y ago

> "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." Just because it's a quote doesn't make it true. Whether a submarine can swim is semantics, yes. Whether something non organic, created by the human can think is groundbreaking and extremely interesting, both for the implications it has, and for the questions we must answer to decide it.