4 ms·

> It seems these godlike AIs are already frighteningly cheap and easy to replicate. "godlike"? Really? I'm not religious, but this seems like an overreaction f

> It seems these godlike AIs are already frighteningly cheap and easy to replicate. "godlike"? Really? I'm not religious, but this seems like an overreaction for something that has no agency.

Jcowell 4y ago
What if creation was a result of a lucky happen-by-chance hallucination ?
- cjohnson318 4y ago
  Sure! Why not? Quantum foam imagining quantum foam. I love it. I still wouldn't consider an LLM god-like. I mean, if it eats its own son, then maybe. (Titan of a reference there.)
crazygringo 4y ago
If it's a shorthand for omniscience then I can see how it makes sense. A bit hyperbolic though for sure.
doctor_eval 4y ago
How do you know agency is not simply the output of a large language model encoded in neurons? What is the difference between neuronal and digital weights? Considering that we don’t know how the brain works so well, and we don’t understand why LLMs work so well, simply on the basis of their output I think the safest assumption is that these models do indeed have agency, or at least the capability of agency.
- cjohnson318 4y ago
  I'm using agency in this context to mean: (1) a strong desire to achieve one or more clear goals, beyond survival, and (2) taking concrete steps to achieve those goals. > How do you know agency is not simply the output of a large language model encoded in neurons? I'm not sure what you mean here. Is agency an emergent effect of large digital or biological neural network? Maybe! Is it an emergent effect of a large language model? If it is, then it should be clear, or demonstrable, that the model (1) has goals (2) takes concrete steps to achieve those goals. > What is the difference between neuronal and digital weights? Brain chemistry works at orders of magnitude less speed, since we're talking about periodically building and releasing an ionic differential between the inside and outside of a cell wall. Moreover, we have a massive number of neurons and a stupidly massive amount of interneuronal connections, with billions of years of training over billions of lineages. Digital weights, in contrast, are a stripped down model of this system that throws out a whole class of complexities like hormones and metabolism. > I think the safest assumption is that these models do indeed have agency, or at least the capability of agency. I think this is an overly generous assumption.
  - doctor_eval 4y ago
    I think the precautionary principle should apply to AI. Assuming and acting as if they have agency, is safer than assuming they don’t. > it should be clear, or demonstrable, that the model (1) has goals (2) takes concrete steps to achieve those goals. That definition seems arbitrary to me; many humans wouldn’t pass this test. On the other hand, LLMs certainly seem capable of acting towards specific goals (such as helpfulness). So, I would say that based on your definition, LLMs have agency. But I think you really meant, internally generated goals. Time will tell. That said, humans who don’t have clear goals can and are coerced into all sorts of damaging behaviours by those who do have goals. So even if I accept that LLMs don’t have their own goals, they can certainly be manipulated to act in favour of the goals of others. That’s effectively what prompt engineering is all about. So I just think it’s a mistake to make assumptions about these LLMs. We don’t know why they work so well, and it will take a very long time until we do. In the meantime, let’s not make assumptions that we can’t justify.
    - cjohnson318 4y ago
      > That definition seems arbitrary to me; many humans wouldn’t pass this test. What part of this is arbitrary, i.e. random, whimsical, or biased? This is a fairly comprehensive working definition of "agency". "Internally generated", which is implied, is a nice touch. Nearly every human over the age of 18 months passes this test with flying colors. Toddlers have goals, and do everything in their power to achieve them. What humans are you thinking of that don't have agency? > In the meantime, let’s not make assumptions that we can’t justify. I totally agree. I think that until demonstrated otherwise, I will assume that LLMs are a giant statistical sieves that (1) periodically spit out text directly from their training set, unmodified, and that (2) do not learn on their own, do not formulate their own goals, and do not take actions to achieve those goals.
      - doctor_eval 4y ago
        Got it. So you assume these systems are inherently safe. What could go wrong?
- ComposedPattern 4y ago
  Gods don't have persistent bodies or brains. They're spirits. I don't disagree that digital systems with neural architectures could have agency in principle, but agency generally is definitely not the output of a large language model. Animals without language have agency, in that that they take actions to fulfill their desires. Current LLMs may have some degree of intelligence, but they don't even appear to have any consistent wishes or desires. You can get them to talk longingly about x... until you give another prompt and suddenly x doesn't matter to them at all.

> It seems these godlike AIs are already frighteningly cheap and easy to replicate. "godlike"? Really? I'm not religious, but this seems like an overreaction for something that has no agency.

Jcowell 4y ago

What if creation was a result of a lucky happen-by-chance hallucination ?

cjohnson318 4y ago
Sure! Why not? Quantum foam imagining quantum foam. I love it. I still wouldn't consider an LLM god-like. I mean, if it eats its own son, then maybe. (Titan of a reference there.)

crazygringo 4y ago

If it's a shorthand for omniscience then I can see how it makes sense. A bit hyperbolic though for sure.

doctor_eval 4y ago

How do you know agency is not simply the output of a large language model encoded in neurons? What is the difference between neuronal and digital weights? Considering that we don’t know how the brain works so well, and we don’t understand why LLMs work so well, simply on the basis of their output I think the safest assumption is that these models do indeed have agency, or at least the capability of agency.

cjohnson318 4y ago
I'm using agency in this context to mean: (1) a strong desire to achieve one or more clear goals, beyond survival, and (2) taking concrete steps to achieve those goals. > How do you know agency is not simply the output of a large language model encoded in neurons? I'm not sure what you mean here. Is agency an emergent effect of large digital or biological neural network? Maybe! Is it an emergent effect of a large language model? If it is, then it should be clear, or demonstrable, that the model (1) has goals (2) takes concrete steps to achieve those goals. > What is the difference between neuronal and digital weights? Brain chemistry works at orders of magnitude less speed, since we're talking about periodically building and releasing an ionic differential between the inside and outside of a cell wall. Moreover, we have a massive number of neurons and a stupidly massive amount of interneuronal connections, with billions of years of training over billions of lineages. Digital weights, in contrast, are a stripped down model of this system that throws out a whole class of complexities like hormones and metabolism. > I think the safest assumption is that these models do indeed have agency, or at least the capability of agency. I think this is an overly generous assumption.
- doctor_eval 4y ago
  I think the precautionary principle should apply to AI. Assuming and acting as if they have agency, is safer than assuming they don’t. > it should be clear, or demonstrable, that the model (1) has goals (2) takes concrete steps to achieve those goals. That definition seems arbitrary to me; many humans wouldn’t pass this test. On the other hand, LLMs certainly seem capable of acting towards specific goals (such as helpfulness). So, I would say that based on your definition, LLMs have agency. But I think you really meant, internally generated goals. Time will tell. That said, humans who don’t have clear goals can and are coerced into all sorts of damaging behaviours by those who do have goals. So even if I accept that LLMs don’t have their own goals, they can certainly be manipulated to act in favour of the goals of others. That’s effectively what prompt engineering is all about. So I just think it’s a mistake to make assumptions about these LLMs. We don’t know why they work so well, and it will take a very long time until we do. In the meantime, let’s not make assumptions that we can’t justify.
  - cjohnson318 4y ago
    > That definition seems arbitrary to me; many humans wouldn’t pass this test. What part of this is arbitrary, i.e. random, whimsical, or biased? This is a fairly comprehensive working definition of "agency". "Internally generated", which is implied, is a nice touch. Nearly every human over the age of 18 months passes this test with flying colors. Toddlers have goals, and do everything in their power to achieve them. What humans are you thinking of that don't have agency? > In the meantime, let’s not make assumptions that we can’t justify. I totally agree. I think that until demonstrated otherwise, I will assume that LLMs are a giant statistical sieves that (1) periodically spit out text directly from their training set, unmodified, and that (2) do not learn on their own, do not formulate their own goals, and do not take actions to achieve those goals.
    - doctor_eval 4y ago
      Got it. So you assume these systems are inherently safe. What could go wrong?
ComposedPattern 4y ago
Gods don't have persistent bodies or brains. They're spirits. I don't disagree that digital systems with neural architectures could have agency in principle, but agency generally is definitely not the output of a large language model. Animals without language have agency, in that that they take actions to fulfill their desires. Current LLMs may have some degree of intelligence, but they don't even appear to have any consistent wishes or desires. You can get them to talk longingly about x... until you give another prompt and suddenly x doesn't matter to them at all.