3 ms·

>Humans don't receive infinite rewards Maybe you're right, but I don't think it's as obvious as you imply. Consider this question. Can there exist hypothetica

by xamuel 6y ago

>Humans don't receive infinite rewards Maybe you're right, but I don't think it's as obvious as you imply. Consider this question. Can there exist hypothetical rewards x1,x2,x3,..., each one of which is significantly better than the previous, and at the same time another hypothetical reward y such that y is significantly better than each x_i? If such rewards can exist, and if "significantly better" implies "at least +1 better", then that necessarily means y must be infinite. Now, how would you argue that such rewards can't exist? If you say, "because then y would have to be infinite", then you're arguing in circles, assuming what you want to prove in the first place. It might well be that such rewards indeed can't exist, but if so, the argument would have to be different, and I suspect nontrivial.

lostmsu 6y ago
Another problem with your paper is related to this one: AGI is defined as a program, that can solve any task a human can. You haven't showed, that humans are capable of solving arbitrary tasks with transfinite rewards, therefore you have not demonstrated, that (even potential) inability to solve them within current framework implies inability to create AGI in it.
PeterisP 6y ago
Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. Human-equivalent minds can obviously be implemented on top a framework that does not have inherently infinite or infinitely divisible values, because human minds are implemented on a top of a substrate that uses a finite amount of discrete neurotransmitters to do everything, and manages to work just fine. If human minds can reason about infinite concepts using "hardware" where all the neural signals used to represent that concept are finite, then why should it be impossible a reinforcement learning system learn to reason about infinite concepts while using only finite numbers to form that representation?
- xamuel 6y ago
  >Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. This is like saying computers can't represent infinity because they have only finitely many bytes. Suppose the treasury rewarded you a "superdollar", which is a special object that allows you to create any number of dollars that you want, on demand, as many times as you want. How many dollars would you say this superdollar is worth? Obviously, no finite number of dollars would be worth that one superdollar. The human mind can certainly understand the relative value of a superdollar vs. any number of dollars. That the human mind is implemented through finite electrochemical processes is irrelevant.
  - PeterisP 6y ago
    The worth of a superdollar is equal to as much dollars as you can spend per second times the seconds you can expect to live. Both numbers are large but finite, the first is bounded by the value of global economy (no matter how many dollars you create, the total purchasing power of these dollars can't grow beyond that), the second is bounded if not to a few hundred years then by the time until the heat death of the universe. In a similar manner, the total amount of reward that you might ever get is capped by the amount of pleasure your brain can perceive at any given moment (which is finite) times your lifespan (which is finite). You might reason and hypothesise about agents perceiving infinite rewards (as we are doing now), but this has nothing to do with the reality of homo sapiens rewards system(s), or, in fact, the rewards system of any agent existing in our physical reality, which is effectively bounded both in time and space.

by xamuel 6y ago

lostmsu 6y ago

Another problem with your paper is related to this one: AGI is defined as a program, that can solve any task a human can. You haven't showed, that humans are capable of solving arbitrary tasks with transfinite rewards, therefore you have not demonstrated, that (even potential) inability to solve them within current framework implies inability to create AGI in it.

PeterisP 6y ago

Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. Human-equivalent minds can obviously be implemented on top a framework that does not have inherently infinite or infinitely divisible values, because human minds are implemented on a top of a substrate that uses a finite amount of discrete neurotransmitters to do everything, and manages to work just fine. If human minds can reason about infinite concepts using "hardware" where all the neural signals used to represent that concept are finite, then why should it be impossible a reinforcement learning system learn to reason about infinite concepts while using only finite numbers to form that representation?

xamuel 6y ago
>Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. This is like saying computers can't represent infinity because they have only finitely many bytes. Suppose the treasury rewarded you a "superdollar", which is a special object that allows you to create any number of dollars that you want, on demand, as many times as you want. How many dollars would you say this superdollar is worth? Obviously, no finite number of dollars would be worth that one superdollar. The human mind can certainly understand the relative value of a superdollar vs. any number of dollars. That the human mind is implemented through finite electrochemical processes is irrelevant.
- PeterisP 6y ago
  The worth of a superdollar is equal to as much dollars as you can spend per second times the seconds you can expect to live. Both numbers are large but finite, the first is bounded by the value of global economy (no matter how many dollars you create, the total purchasing power of these dollars can't grow beyond that), the second is bounded if not to a few hundred years then by the time until the heat death of the universe. In a similar manner, the total amount of reward that you might ever get is capped by the amount of pleasure your brain can perceive at any given moment (which is finite) times your lifespan (which is finite). You might reason and hypothesise about agents perceiving infinite rewards (as we are doing now), but this has nothing to do with the reality of homo sapiens rewards system(s), or, in fact, the rewards system of any agent existing in our physical reality, which is effectively bounded both in time and space.