4 ms·

> Can AI apply knowledge learned in one topic to another topic We don't have AGI yet. and AGI will have deception just about by definition anyway, so the point

by platz 2y ago

> Can AI apply knowledge learned in one topic to another topic We don't have AGI yet. and AGI will have deception just about by definition anyway, so the point is moot.

TrainedMonkey 2y ago
> We don't have AGI yet. How do you know? I think the best we can say is that we probably don't have AGI yet. If an entity achieves the AGI the optimal strategy could quite well be: 1. Don't tell anyone and amass wealth through trading algorithms, patents, etc. 2. Once there are indications that someone else will achieve AGI soon - go public with yours to capture market.
- platz 2y ago
  I don't believe we live in such epistemic black holes. how do you know you're not a brain in a vat. all we know is that we probably aren't a brain in a vat. hah.
  - hnfong 2y ago
    > I don't believe we live in such epistemic black holes That's fine. > how do you know you're not a brain in a vat. I don't. > all we know is that we probably aren't a brain in a vat. hah. No we don't, because I don't know. You're free to believe whatever though, just don't speak for everyone else.
    - stale2002 2y ago
      > No we don't, because I don't know. The problem here is that people are making claims to the real world with this kind of stuff. They are advocating that we aught "do something" about this. But, if you retreat to such unfalsifiable claims, you are basically removing yourself from all normal scientific claims that govern all parts of the rest of the world. To go back to the brain in the vat example, imagine someone was advocating for new laws to govern this brain in a vat theory. I would hope that you wouldn't support some significant legal change to society, merely because you "dont know" that we aren't brains in a vat.
      - bumby 2y ago
        >normal scientific claims that govern all parts of the rest of the world. Isn’t is possible that the problem can’t be answered by science? E.g., if their central question is about consciousness, science may not be equipped to answer it. Science is concerned with objective evidence, and the hard problem of conscience deals with subjective experience.
        stale2002 2y ago
        > Isn’t is possible that the problem can’t be answered by science? If you definitionally can't answer the question then I am not sure why people who are living in the real world should spend much time on it. So my point stands. Unfalsifiable claims that are "not even wrong" aren't particularly useful when discussing actual things of importance, like what kinds of laws we should make. Keep that stuff to the introductory college philosophy classes, while the rest of us work on the stuff that actually matters.
        bumby 2y ago
        Please elaborate on how you define what “actually matters.” It seems to me that subjective experience is one of those things, if you value experience at all. If you don’t, I’m not sure anything matters. And there’s plenty both within and without science that can’t (currently, at least) be defined. But it doesn’t mean they aren’t worth probing.
        deleted 2y ago
        [deleted]
        deleted 2y ago
        [deleted]
- stale2002 2y ago
  Its funny the best argument that AGI doomers can make is age old unfalsifiable claims. "how can you disprove the existence of a magic, infinitely powerful god! It can just magic away the evidence." To quote the rationalists. Your statement is "not even wrong!". Coming up with increasing tenuous excuses for why we can ignore the commonly accepted rules of evidence isn't the way to win an argument on this stuff.
  - pixl97 2y ago
    Conversely, how many monkeys control man versus the counter of how many men control monkeys. This is something that is observable and countable. Giving a system intelligence gives it power, we are the gods in the monkeys eyes.
- rebolek 2y ago
  If there's AGI, it would decide what to do, it wouldn't be commanded by someone. That's the definition of general intelligence, it can make decisions.
  - TrainedMonkey 2y ago
    You might be thinking of super intelligence which we cannot control by default as it would outsmart any control systems. Note, that is not to say it would not cooperate - it could very well choose to follow the parameters given to it. For non super intelligences there are plenty of example of intelligences controlling other intelligences through various means: force, power structures, economic system, etc.
- not2b 2y ago
  Occam's razor. AGI isn't necessary to explain the observed behavior; a model that has no concept of truth or falsehood, but is just optimized to achieve goals will engage in this kind of behavior. If someone with the needed resources decides to train models with the goal of amassing wealth, those models, without any real intelligence and knowledge about the world other than text and a score representing current net worth, will come up with deceptive strategies, especially when it has the whole Internet to learn about possible strategies.

by platz 2y ago

> Can AI apply knowledge learned in one topic to another topic We don't have AGI yet. and AGI will have deception just about by definition anyway, so the point is moot.

TrainedMonkey 2y ago

> We don't have AGI yet. How do you know? I think the best we can say is that we probably don't have AGI yet. If an entity achieves the AGI the optimal strategy could quite well be: 1. Don't tell anyone and amass wealth through trading algorithms, patents, etc. 2. Once there are indications that someone else will achieve AGI soon - go public with yours to capture market.

platz 2y ago
I don't believe we live in such epistemic black holes. how do you know you're not a brain in a vat. all we know is that we probably aren't a brain in a vat. hah.
- hnfong 2y ago
  > I don't believe we live in such epistemic black holes That's fine. > how do you know you're not a brain in a vat. I don't. > all we know is that we probably aren't a brain in a vat. hah. No we don't, because I don't know. You're free to believe whatever though, just don't speak for everyone else.
  - stale2002 2y ago
    > No we don't, because I don't know. The problem here is that people are making claims to the real world with this kind of stuff. They are advocating that we aught "do something" about this. But, if you retreat to such unfalsifiable claims, you are basically removing yourself from all normal scientific claims that govern all parts of the rest of the world. To go back to the brain in the vat example, imagine someone was advocating for new laws to govern this brain in a vat theory. I would hope that you wouldn't support some significant legal change to society, merely because you "dont know" that we aren't brains in a vat.
    - bumby 2y ago
      >normal scientific claims that govern all parts of the rest of the world. Isn’t is possible that the problem can’t be answered by science? E.g., if their central question is about consciousness, science may not be equipped to answer it. Science is concerned with objective evidence, and the hard problem of conscience deals with subjective experience.
      - stale2002 2y ago
        > Isn’t is possible that the problem can’t be answered by science? If you definitionally can't answer the question then I am not sure why people who are living in the real world should spend much time on it. So my point stands. Unfalsifiable claims that are "not even wrong" aren't particularly useful when discussing actual things of importance, like what kinds of laws we should make. Keep that stuff to the introductory college philosophy classes, while the rest of us work on the stuff that actually matters.
        bumby 2y ago
        Please elaborate on how you define what “actually matters.” It seems to me that subjective experience is one of those things, if you value experience at all. If you don’t, I’m not sure anything matters. And there’s plenty both within and without science that can’t (currently, at least) be defined. But it doesn’t mean they aren’t worth probing.
        deleted 2y ago
        [deleted]
        deleted 2y ago
        [deleted]
stale2002 2y ago
Its funny the best argument that AGI doomers can make is age old unfalsifiable claims. "how can you disprove the existence of a magic, infinitely powerful god! It can just magic away the evidence." To quote the rationalists. Your statement is "not even wrong!". Coming up with increasing tenuous excuses for why we can ignore the commonly accepted rules of evidence isn't the way to win an argument on this stuff.
- pixl97 2y ago
  Conversely, how many monkeys control man versus the counter of how many men control monkeys. This is something that is observable and countable. Giving a system intelligence gives it power, we are the gods in the monkeys eyes.
rebolek 2y ago
If there's AGI, it would decide what to do, it wouldn't be commanded by someone. That's the definition of general intelligence, it can make decisions.
- TrainedMonkey 2y ago
  You might be thinking of super intelligence which we cannot control by default as it would outsmart any control systems. Note, that is not to say it would not cooperate - it could very well choose to follow the parameters given to it. For non super intelligences there are plenty of example of intelligences controlling other intelligences through various means: force, power structures, economic system, etc.
not2b 2y ago
Occam's razor. AGI isn't necessary to explain the observed behavior; a model that has no concept of truth or falsehood, but is just optimized to achieve goals will engage in this kind of behavior. If someone with the needed resources decides to train models with the goal of amassing wealth, those models, without any real intelligence and knowledge about the world other than text and a score representing current net worth, will come up with deceptive strategies, especially when it has the whole Internet to learn about possible strategies.