5 ms·

I do wonder about the ethical issues that people are going to start facing in, I don't know, maybe 10-20 years? Maybe even sooner. These new DNNs create very h

I do wonder about the ethical issues that people are going to start facing in, I don't know, maybe 10-20 years? Maybe even sooner. These new DNNs create very human like outputs. And their structure is rather similar to the brain. Now we are learning that maybe they're even more brainlike than we thought. At what point do we cross a threshold here and encounter a non-trivial number of people who argue that a sufficiently large model actually is a brain, and therefore deserves rights? Blake Lemoine went there already but I wonder if that's going to be an isolated incident or a harbinger of things to come. It feels weird. On one hand, I know that these things are just programs. On the other hand, I also know that our brains are just molecules and cells. At the same time I feel that intelligent creatures deserve rights and the exact way the underlying brain works doesn't seem especially important. Especially if people start getting brain augmentations at scale a la Neuralink. At what point do you say the line is crossed?

heyitsguay 4y ago
There are no actual ethical issues with contemporary ML architectures (including transformers) being too conscious or brainlike, it's just laypeople and demagogue chatter. It could be an issue in the future but only with computational systems that are utterly unlike the ones being created and used today. Actual practitioners are tired of the debate because getting to an informed answer requires grokking some undergraduate-level math and statistics, and nobody seems particularly willing to do that
- data_maan 4y ago
  > nobody seems particularly willing to do that This. The amount of mathematical illiteracy is staggering in ML.
  - heyitsguay 4y ago
    The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics. It's more the people who talk a lot about AI in grandiose and anthropomorphized terms that I was referring to.
    - data_maan 4y ago
      I work at one of the best unis in the world in a big ML research group and I have not. Unfortunately. I even know researchers with 10k+ citations in ML that even talk about "continuous maths" and "discrete maths". This pretty much sums up their level of mathematical sophistication and ability.
      - heyitsguay 4y ago
        What do you mean? That's an incredibly important distinction in understanding mathematics for ML in the neural net age. Perhaps a bit of a sensitive spot for me personally, coming from a harmonic analysis group for my math PhD, but the short version basically goes like: Up until the 2010s or so, a huge portion of applied math was driven by results from "continuous math": functions mostly take values in some unbounded continuous vector space, they're infinite-dimensional objects supported on some continuous subset of R^n or C^n or whatever, and we reason about signal processing by proving results about existence, uniqueness, and optimality of solutions to certain optimization problems. The infinite-dimensional function spaces provide intellectual substance and challenge to the approach, while also being limited in applicability to circumstances amenable to the many assumptions one must typically make about a signal or sensing mechanism in order for the math model to apply. This is all well and good, but it's a heavy price to pay for what is, essentially, an abstraction. There are no infinities, computed (not computable) functions are really just finite-dimensional vectors taking values in a bounded range, any relationships between domain elements are discrete. In this circumstance, most of the traditional heavy-duty mathematical machinery for signal processing is irrelevant -- equation solutions trivially exist (or don't) and the actual meat of the problem is efficiently computing solutions (or approximate solutions). It's still quite technical and relies on advanced math, but a different sort from what is classically the "aesthetic" higher math approach. Famously, it also means far fewer proofs! At least as apply to real-world applications. The parameter spaces are so large, the optimization landscapes so complicated, that traditional methods don't offer much insight, though people continue to work on it. So now we're just concerned with entirely different problems requiring different tools. Without any further context, that's what I would assume your colleague was referring to, as this is a well-understood mathematical dichotomy.
- abeppu 4y ago
  I don't think the undergrad math and statistics are really the problem. I think even if people understood the math, there would be some who questioned whether there's something epiphenomenon arising from the math, because there are more fundamental issues. Students in philosophy of mind classes were confronting these issues long before the DL boom started, with the Chinese Room argument. I.e. even if we accept as premise that there's a pre-described and deterministic set of steps that gives rise to seemingly intelligent behavior, we can be divided over what properties to ascribe to that system. Does it "understand"? Is it "thinking"? Is it a "person"? Why? - we've never settled on a definition of consciousness - we don't understand how biological brains create qualia - we're not entirely in agreement over when human biological brains stop having ethical standing - we disagree about how necessary embodiment or the material world outside the brain is to "mind" - we disagree on what ethical standing to give to non-human animals with biological brains Because we know so little, perhaps the only people that can believe strongly that current ML models are conscious have embraced panpsychism. But also because we know so little, including _what consciousness is_, I also don't think one can be confident that no ML model today has any consciousness whatsoever. Recently I was re-reading some Douglas Hofstadter and in Metamagical Themas he has a dialogue about the "Careenium" in which on different (time)scales a physical system of balls and barriers appears to have qualitatively different kinds of dynamics. Hofstadter earnestly wants to use this to explain how "I" can direct or constrain in a top-down way some of the physical mechanisms that make me up while at the same time in a bottom-up way they comprise and animate me. I don't know if it was intentional, but it seems awfully similar to a beer-can apparatus that Searle somewhat dismissively uses as example of a potential computing substrate which could surely never support a "mind" no matter how it is programmed. Two academics, both well respected, both who spent decades thinking about the mind and how it comes to be and how we should think about it, and both used an inscrutable complex physical (non-biological) apparatus to make points which are diametrically opposed about how the mind arises from more or less specific physical conditions. I have in mind a property fwomp which I can't clearly define. But I have fwomp, and I'm pretty sure you have fwomp. I don't know where fwomp comes from, how it works or what are the necessary conditions for its persistence. From the preceding sentences, can you know with certainty that U-Nets don't have fwomp?
  - data_maan 4y ago
- origin_path 4y ago
  That's not really an argument, is it? I do understand the maths behind these models, or at least a good chunk of it. I can talk about differentiable functions, softmax etc. Regardless, going from "it involves maths" to "therefore there are no actual ethical issues" is a non-sequitur. Your conclusion doesn't follow from your premise. I actually agree with you - I don't think current transformer models like DALL-E are conscious or worthy of rights - but what you've presented here isn't something that could possibly convince anyone in either direction. You'd have to lay out why it's not and how many years of study of maths are involved really doesn't seem like a convincing basis on which to decide.
ok_dad 4y ago
I personally think we'll have the opposite problem. I think making machines too much like humans will result in less ethical consideration of actual humans, not extra rights for machines. Once a model becomes sufficiently like a human brain to perform many of the jobs we have in our society (driving vehicles, collecting trash, monitoring cameras and data streams, etc.) then those who have the ultimate power in our world will start to see humans as unimportant meat sacks that are inefficient compared to the machines. They'll stop pretending they care about the welfare of humanity even a little bit, and will start to push for policies that would reduce our numbers. Eventually, I believe the end goal will be for a small number of humans to do those jobs that simply cannot be automated, and to serve an even smaller number of masters who control the machines, which will then be purposed as the police for the other humans.
- behnamoh 4y ago
  That's like an interesting movie/show plot that I'd love to watch!
- kevinventullo 4y ago
  You write in the future tense, but I think that this is already happening. Fewer millenials are having children than previous generations, and those that are have fewer children. I think the main driver is that most millenials simply can’t afford children, and this is a direct result of policies pushed by “those who have the ultimate power”.
  - Noumenon72 4y ago
    Or maybe the millennials themselves see humans as unefficient meat sacks who might as well just pass the time with video games rather than striving. Or maybe people actually think every child is precious, and those who have the ultimate power benevolently used their power to require every child get the resources it deserves, even though we can't afford it.
    - kevinventullo 4y ago
      Ah, so the “essential workers” making minimum wage don’t deserve to have kids because they should have been striving more. I see the plan is working! Also, if those in power require every child get the resources it deserves, doesn’t that make it easier to have kids?
aaaaaaaaaaab 4y ago
>At what point do we cross a threshold here and encounter a non-trivial number of people who argue that a sufficiently large model actually is a brain, and therefore deserves rights? There are two ways of resolving this conundrum. You either give rights to computer programs, or take away the rights of people. Which option sounds more likely to you?
petra 4y ago
At it's heart, ethics isn't a cognitive process, but it's based on an emotional one. We care about domesticated animals because of Oxytocin(OT), the love/connection hormone: " Recent reports have indicated the possible contribution of OT to the formation of a social bond between domesticated mammals (dog, sheep, cattle) and humans."[1] And sure, we'll probably create at some point an artificial creature that will release oxytocin in humans. An it's an interesting question how to design such a machine. But most AI's ? most likely, we won't feel any strong connection with them. [1]https://link.springer.com/article/10.1134/S2079059717030042 https://link.springer.com/article/10.1134/S2079059717030042
- hooverd 4y ago
  They're called vtubers.
- origin_path 4y ago
  Mmm, perhaps. I would be very warey of any study of the form "possible contribution of molecule X to high level social process Y", that sounds very far into the junk science danger zone. Dogs came from wolves, originally. Nothing cute or love inducing about those. They were domesticated because they were useful and the warm feelings came later. Sheep, cattle. We raise them for wool, milk and food. It's a purely transactional form of dominance. Very few people would profess love for a cow or a sheep outside of maybe a children's petting zoo.
quonn 4y ago
Feeling creatures deserve rights, since they can suffer. Intelligence is at best indirectly related, if at all.
- visarga 4y ago
  I think feeling is being approximated in reinforcement learning by assigning values to states, it's an important part of selecting the next action. Emotion is related to the expected reward from the current state. So an RL agent implemented with neural nets would deserve rights because it has a concept of what is good and bad vs. its goals.

heyitsguay 4y ago

There are no actual ethical issues with contemporary ML architectures (including transformers) being too conscious or brainlike, it's just laypeople and demagogue chatter. It could be an issue in the future but only with computational systems that are utterly unlike the ones being created and used today. Actual practitioners are tired of the debate because getting to an informed answer requires grokking some undergraduate-level math and statistics, and nobody seems particularly willing to do that

data_maan 4y ago
> nobody seems particularly willing to do that This. The amount of mathematical illiteracy is staggering in ML.
- heyitsguay 4y ago
  The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics. It's more the people who talk a lot about AI in grandiose and anthropomorphized terms that I was referring to.
  - data_maan 4y ago
    I work at one of the best unis in the world in a big ML research group and I have not. Unfortunately. I even know researchers with 10k+ citations in ML that even talk about "continuous maths" and "discrete maths". This pretty much sums up their level of mathematical sophistication and ability.
    - heyitsguay 4y ago
      What do you mean? That's an incredibly important distinction in understanding mathematics for ML in the neural net age. Perhaps a bit of a sensitive spot for me personally, coming from a harmonic analysis group for my math PhD, but the short version basically goes like: Up until the 2010s or so, a huge portion of applied math was driven by results from "continuous math": functions mostly take values in some unbounded continuous vector space, they're infinite-dimensional objects supported on some continuous subset of R^n or C^n or whatever, and we reason about signal processing by proving results about existence, uniqueness, and optimality of solutions to certain optimization problems. The infinite-dimensional function spaces provide intellectual substance and challenge to the approach, while also being limited in applicability to circumstances amenable to the many assumptions one must typically make about a signal or sensing mechanism in order for the math model to apply. This is all well and good, but it's a heavy price to pay for what is, essentially, an abstraction. There are no infinities, computed (not computable) functions are really just finite-dimensional vectors taking values in a bounded range, any relationships between domain elements are discrete. In this circumstance, most of the traditional heavy-duty mathematical machinery for signal processing is irrelevant -- equation solutions trivially exist (or don't) and the actual meat of the problem is efficiently computing solutions (or approximate solutions). It's still quite technical and relies on advanced math, but a different sort from what is classically the "aesthetic" higher math approach. Famously, it also means far fewer proofs! At least as apply to real-world applications. The parameter spaces are so large, the optimization landscapes so complicated, that traditional methods don't offer much insight, though people continue to work on it. So now we're just concerned with entirely different problems requiring different tools. Without any further context, that's what I would assume your colleague was referring to, as this is a well-understood mathematical dichotomy.
abeppu 4y ago
I don't think the undergrad math and statistics are really the problem. I think even if people understood the math, there would be some who questioned whether there's something epiphenomenon arising from the math, because there are more fundamental issues. Students in philosophy of mind classes were confronting these issues long before the DL boom started, with the Chinese Room argument. I.e. even if we accept as premise that there's a pre-described and deterministic set of steps that gives rise to seemingly intelligent behavior, we can be divided over what properties to ascribe to that system. Does it "understand"? Is it "thinking"? Is it a "person"? Why? - we've never settled on a definition of consciousness - we don't understand how biological brains create qualia - we're not entirely in agreement over when human biological brains stop having ethical standing - we disagree about how necessary embodiment or the material world outside the brain is to "mind" - we disagree on what ethical standing to give to non-human animals with biological brains Because we know so little, perhaps the only people that can believe strongly that current ML models are conscious have embraced panpsychism. But also because we know so little, including _what consciousness is_, I also don't think one can be confident that no ML model today has any consciousness whatsoever. Recently I was re-reading some Douglas Hofstadter and in Metamagical Themas he has a dialogue about the "Careenium" in which on different (time)scales a physical system of balls and barriers appears to have qualitatively different kinds of dynamics. Hofstadter earnestly wants to use this to explain how "I" can direct or constrain in a top-down way some of the physical mechanisms that make me up while at the same time in a bottom-up way they comprise and animate me. I don't know if it was intentional, but it seems awfully similar to a beer-can apparatus that Searle somewhat dismissively uses as example of a potential computing substrate which could surely never support a "mind" no matter how it is programmed. Two academics, both well respected, both who spent decades thinking about the mind and how it comes to be and how we should think about it, and both used an inscrutable complex physical (non-biological) apparatus to make points which are diametrically opposed about how the mind arises from more or less specific physical conditions. I have in mind a property fwomp which I can't clearly define. But I have fwomp, and I'm pretty sure you have fwomp. I don't know where fwomp comes from, how it works or what are the necessary conditions for its persistence. From the preceding sentences, can you know with certainty that U-Nets don't have fwomp?
- data_maan 4y ago
origin_path 4y ago
That's not really an argument, is it? I do understand the maths behind these models, or at least a good chunk of it. I can talk about differentiable functions, softmax etc. Regardless, going from "it involves maths" to "therefore there are no actual ethical issues" is a non-sequitur. Your conclusion doesn't follow from your premise. I actually agree with you - I don't think current transformer models like DALL-E are conscious or worthy of rights - but what you've presented here isn't something that could possibly convince anyone in either direction. You'd have to lay out why it's not and how many years of study of maths are involved really doesn't seem like a convincing basis on which to decide.

ok_dad 4y ago

I personally think we'll have the opposite problem. I think making machines too much like humans will result in less ethical consideration of actual humans, not extra rights for machines. Once a model becomes sufficiently like a human brain to perform many of the jobs we have in our society (driving vehicles, collecting trash, monitoring cameras and data streams, etc.) then those who have the ultimate power in our world will start to see humans as unimportant meat sacks that are inefficient compared to the machines. They'll stop pretending they care about the welfare of humanity even a little bit, and will start to push for policies that would reduce our numbers. Eventually, I believe the end goal will be for a small number of humans to do those jobs that simply cannot be automated, and to serve an even smaller number of masters who control the machines, which will then be purposed as the police for the other humans.

behnamoh 4y ago
That's like an interesting movie/show plot that I'd love to watch!
kevinventullo 4y ago
You write in the future tense, but I think that this is already happening. Fewer millenials are having children than previous generations, and those that are have fewer children. I think the main driver is that most millenials simply can’t afford children, and this is a direct result of policies pushed by “those who have the ultimate power”.
- Noumenon72 4y ago
  Or maybe the millennials themselves see humans as unefficient meat sacks who might as well just pass the time with video games rather than striving. Or maybe people actually think every child is precious, and those who have the ultimate power benevolently used their power to require every child get the resources it deserves, even though we can't afford it.
  - kevinventullo 4y ago
    Ah, so the “essential workers” making minimum wage don’t deserve to have kids because they should have been striving more. I see the plan is working! Also, if those in power require every child get the resources it deserves, doesn’t that make it easier to have kids?

aaaaaaaaaaab 4y ago

>At what point do we cross a threshold here and encounter a non-trivial number of people who argue that a sufficiently large model actually is a brain, and therefore deserves rights? There are two ways of resolving this conundrum. You either give rights to computer programs, or take away the rights of people. Which option sounds more likely to you?

petra 4y ago

At it's heart, ethics isn't a cognitive process, but it's based on an emotional one. We care about domesticated animals because of Oxytocin(OT), the love/connection hormone: " Recent reports have indicated the possible contribution of OT to the formation of a social bond between domesticated mammals (dog, sheep, cattle) and humans."[1] And sure, we'll probably create at some point an artificial creature that will release oxytocin in humans. An it's an interesting question how to design such a machine. But most AI's ? most likely, we won't feel any strong connection with them. [1]https://link.springer.com/article/10.1134/S2079059717030042 https://link.springer.com/article/10.1134/S2079059717030042

hooverd 4y ago
They're called vtubers.
origin_path 4y ago
Mmm, perhaps. I would be very warey of any study of the form "possible contribution of molecule X to high level social process Y", that sounds very far into the junk science danger zone. Dogs came from wolves, originally. Nothing cute or love inducing about those. They were domesticated because they were useful and the warm feelings came later. Sheep, cattle. We raise them for wool, milk and food. It's a purely transactional form of dominance. Very few people would profess love for a cow or a sheep outside of maybe a children's petting zoo.

quonn 4y ago

Feeling creatures deserve rights, since they can suffer. Intelligence is at best indirectly related, if at all.

visarga 4y ago
I think feeling is being approximated in reinforcement learning by assigning values to states, it's an important part of selecting the next action. Emotion is related to the expected reward from the current state. So an RL agent implemented with neural nets would deserve rights because it has a concept of what is good and bad vs. its goals.