4 ms·

Right, I should have clarified that I meant: create a child in such a way as to know the child's truthfulness and source-code. Of course, without that requireme

by xamuel 6y ago

Right, I should have clarified that I meant: create a child in such a way as to know the child's truthfulness and source-code. Of course, without that requirement, a monkey could blindly type up an AGI source-code on a typewrite by sheer dumb luck. As for an AGI simply self-replicating, that is possible but then the AGI would not know the child's truthfulness and source-code, since an AGI cannot know its own truthfulness and source-code.

at_a_remove 6y ago
Still, if it is identical, then it could not necessarily be less than. I may not know what value is in the box (my own truthfulness and source-code), but if it is replicated, then the value ought to be identical, and therefore not less than.
- xamuel 6y ago
  Right, but the machine which prints its own source-code does not do so in such a way as to know the child's truthfulness (because the machine cannot know its own truthfulness and source-code, assuming the machine itself is truthful and satisfies basic assumptions about how it can reason). I could design a program that systematically prints out all possible source-codes, and then try to claim credit for designing the first AGI (because, at least if AGI is possible at all, my brute-force source-code-printer will eventually print it). But that would be silly because I wouldn't actually know which of the many many outputs from my printer is the actual AGI source-code :)
  - at_a_remove 6y ago
    I think the "knowing" is irrelevant. If I have two boxes, and you tell me that the contents, the value of box one is identical to box two, then I do not need to know the value of the box to know that box two is not less than box one. Here is a pointer to a section of memory; copy it. As long as the copy operation is successful, you do not need to know the value of what you have copied to know that it is identical, and therefore not "less than." Hence, if you self-replicate, then it is not "less than;" and if you cannot know that two things are identical, then you cannot self-replicate. I think you are stuck saying that your AGI cannot self-replicate if you want to hold onto "necessarily less than." It's just a basic principle of equality, you cannot get around it.
    - xamuel 6y ago
      I think we're talking past each other. My claim is that if an AGI single-handedly creates a child AGI, in such a way as to know the child's source-code and truthfulness, then the parent is more intelligent than the child (in a certain technical sense). You rightly point out that an AGI can duplicate itself (if it knows its own source-code). This doesn't contradict my claim, because it doesn't satisfy the premise. It's like if I give you a box of candies and I say, "Every blue candy in this box is round", and you pull out a red candy which is square, you can't say my statement was false because that red candy isn't round. To falsify the statement you'd have to produce a blue candy that isn't round. The AGI which self-duplicates would only falsify my claim if it self-duplicates in such a way as to know the duplicate's source-code and truthfulness. Assuming said AGI is truthful and satisfies certain assumptions about how it can reason and know, then it can't know its own source-code and truthfulness, so it doesn't falsify my claim.
      - at_a_remove 6y ago
        I don't think we are talking past each other, I think we're having a debate as to whether or not methods of replication have some impact on whether or not two things are identical, and therefore have the same value. Imagine a simple single-celled organism. It is well-suited for its environment and so forth. It reproduces asexually without error. Now, it isn't terribly bright, it hasn't developed PCR or anything, it hasn't had a Watson or a Crick, so it doesn't "know" its source code, but it has definitely replicated and done so to produce an identical copy. I'm not even sure what truthfulness means in this context, but I can confidently say that identical things have the same value, and therefore the offspring organism not not less than the original. The "in such a way" has an impact in the way you're viewing it, and to me, I do not understand that. Identical is identical. I don't have any other way of looking at that. If the "in such a way" does have some kind of effect, then they are not identical and you can hardly say that the AGI has self-replicated. Can your AGI "quine out" a copy of itself or not? Can I look at the two programs on disc and perform a binary comparison to prove that they are byte-per-byte identical?
        xamuel 6y ago

by xamuel 6y ago

at_a_remove 6y ago

Still, if it is identical, then it could not necessarily be less than. I may not know what value is in the box (my own truthfulness and source-code), but if it is replicated, then the value ought to be identical, and therefore not less than.

xamuel 6y ago
Right, but the machine which prints its own source-code does not do so in such a way as to know the child's truthfulness (because the machine cannot know its own truthfulness and source-code, assuming the machine itself is truthful and satisfies basic assumptions about how it can reason). I could design a program that systematically prints out all possible source-codes, and then try to claim credit for designing the first AGI (because, at least if AGI is possible at all, my brute-force source-code-printer will eventually print it). But that would be silly because I wouldn't actually know which of the many many outputs from my printer is the actual AGI source-code :)
- at_a_remove 6y ago
  I think the "knowing" is irrelevant. If I have two boxes, and you tell me that the contents, the value of box one is identical to box two, then I do not need to know the value of the box to know that box two is not less than box one. Here is a pointer to a section of memory; copy it. As long as the copy operation is successful, you do not need to know the value of what you have copied to know that it is identical, and therefore not "less than." Hence, if you self-replicate, then it is not "less than;" and if you cannot know that two things are identical, then you cannot self-replicate. I think you are stuck saying that your AGI cannot self-replicate if you want to hold onto "necessarily less than." It's just a basic principle of equality, you cannot get around it.
  - xamuel 6y ago
    I think we're talking past each other. My claim is that if an AGI single-handedly creates a child AGI, in such a way as to know the child's source-code and truthfulness, then the parent is more intelligent than the child (in a certain technical sense). You rightly point out that an AGI can duplicate itself (if it knows its own source-code). This doesn't contradict my claim, because it doesn't satisfy the premise. It's like if I give you a box of candies and I say, "Every blue candy in this box is round", and you pull out a red candy which is square, you can't say my statement was false because that red candy isn't round. To falsify the statement you'd have to produce a blue candy that isn't round. The AGI which self-duplicates would only falsify my claim if it self-duplicates in such a way as to know the duplicate's source-code and truthfulness. Assuming said AGI is truthful and satisfies certain assumptions about how it can reason and know, then it can't know its own source-code and truthfulness, so it doesn't falsify my claim.
    - at_a_remove 6y ago
      I don't think we are talking past each other, I think we're having a debate as to whether or not methods of replication have some impact on whether or not two things are identical, and therefore have the same value. Imagine a simple single-celled organism. It is well-suited for its environment and so forth. It reproduces asexually without error. Now, it isn't terribly bright, it hasn't developed PCR or anything, it hasn't had a Watson or a Crick, so it doesn't "know" its source code, but it has definitely replicated and done so to produce an identical copy. I'm not even sure what truthfulness means in this context, but I can confidently say that identical things have the same value, and therefore the offspring organism not not less than the original. The "in such a way" has an impact in the way you're viewing it, and to me, I do not understand that. Identical is identical. I don't have any other way of looking at that. If the "in such a way" does have some kind of effect, then they are not identical and you can hardly say that the AGI has self-replicated. Can your AGI "quine out" a copy of itself or not? Can I look at the two programs on disc and perform a binary comparison to prove that they are byte-per-byte identical?
      - xamuel 6y ago