4 ms·

If the model captures (almost) all information in the training sample, that is memorization. In your example 2x + 3 memorized the point (1,5) because you can re

If the model captures (almost) all information in the training sample, that is memorization. In your example 2x + 3 memorized the point (1,5) because you can recall 5 as f(1).

kilgnad 4y ago
This doesn't make sense. F(X) itself is not (1,5). You have to apply F for it to work. Paint and paint brushes can be thought of as functions. You apply the right inputs (brush strokes) and you get an output that is a painting. Therefore could you say that all paintings in the world are encoded into the paintbrush? No. You can't. Not until you use the paintbrush to copy something.
- WithinReason 4y ago
  If I ask you to recite a poem you're a function f("recite poem") which returns the memorized poem. It's not even possible to have memorisation without a function.
  - kilgnad 4y ago
    But it's not memorized. The "poem" in the case of the machine learning model is constructed from a general algorithm. Similar to how 2x+1 has no concept of 3. When you input a 1, it does a *2 operation and a +1 operation and produces a 3. Nothing is memorized. 3 arises as a side effect of unrelated transformation operations.
    - WithinReason 4y ago
      "*2" and "+1" is stored information that's used produce 5. The memory of a poem is stored information that's used to vibrate your vocal cords and produce a recited poem.
      - kilgnad 4y ago
        The memory of a poem is stored information but your vocal cord is NOT stored information. "*2" and "+1" are vocal cords. They operate on any number of inputs just like how your vocal cord operates on any number of nerve signals and air blowing through it. If my vocal coords are never used to record a copyrighted song then no violation of the law occured. If I never use the machine learning model to generate an existing picture then no law violation occured.
        WithinReason 4y ago
        Well you can get around the whole memorisation problem by only using unique training samples in your dataset, same for Copilot. But yes, I do agree that in this case the user is doing the violation, not the network.
        kilgnad 4y ago
        Unique training samples only makes the problem less frequent, it does not eliminate the problem. The curve can still intersect a unique point.
        WithinReason 4y ago
        It seemed to me all the examples in the article (and in copilot examples as well) are repeated training points, so I think it would solve the problem.
        kilgnad 4y ago
        That just means only for those examples it was fixed. Logically the mathematics behind it says that it can happen. You present evidence which is refutable by new evidence. Logic if correct, cannot be refuted.
        WithinReason 4y ago
    - FeepingCreature 4y ago
      There's a joke programming language, HQ9, that only has three instructions, each expressed with a single character. One outputs "Hello world!", one prints the language's source code (quine), one prints "99 Bottles of Beer...". Does the interpreter have 99 Bottles memorized? After all, it is a function (with a 3-element domain).

If the model captures (almost) all information in the training sample, that is memorization. In your example 2x + 3 memorized the point (1,5) because you can recall 5 as f(1).

kilgnad 4y ago

This doesn't make sense. F(X) itself is not (1,5). You have to apply F for it to work. Paint and paint brushes can be thought of as functions. You apply the right inputs (brush strokes) and you get an output that is a painting. Therefore could you say that all paintings in the world are encoded into the paintbrush? No. You can't. Not until you use the paintbrush to copy something.

WithinReason 4y ago
If I ask you to recite a poem you're a function f("recite poem") which returns the memorized poem. It's not even possible to have memorisation without a function.
- kilgnad 4y ago
  But it's not memorized. The "poem" in the case of the machine learning model is constructed from a general algorithm. Similar to how 2x+1 has no concept of 3. When you input a 1, it does a *2 operation and a +1 operation and produces a 3. Nothing is memorized. 3 arises as a side effect of unrelated transformation operations.
  - WithinReason 4y ago
    "*2" and "+1" is stored information that's used produce 5. The memory of a poem is stored information that's used to vibrate your vocal cords and produce a recited poem.
    - kilgnad 4y ago
      The memory of a poem is stored information but your vocal cord is NOT stored information. "*2" and "+1" are vocal cords. They operate on any number of inputs just like how your vocal cord operates on any number of nerve signals and air blowing through it. If my vocal coords are never used to record a copyrighted song then no violation of the law occured. If I never use the machine learning model to generate an existing picture then no law violation occured.
      - WithinReason 4y ago
        Well you can get around the whole memorisation problem by only using unique training samples in your dataset, same for Copilot. But yes, I do agree that in this case the user is doing the violation, not the network.
        kilgnad 4y ago
        Unique training samples only makes the problem less frequent, it does not eliminate the problem. The curve can still intersect a unique point.
        WithinReason 4y ago
        It seemed to me all the examples in the article (and in copilot examples as well) are repeated training points, so I think it would solve the problem.
        kilgnad 4y ago
        That just means only for those examples it was fixed. Logically the mathematics behind it says that it can happen. You present evidence which is refutable by new evidence. Logic if correct, cannot be refuted.
        WithinReason 4y ago
  - FeepingCreature 4y ago
    There's a joke programming language, HQ9, that only has three instructions, each expressed with a single character. One outputs "Hello world!", one prints the language's source code (quine), one prints "99 Bottles of Beer...". Does the interpreter have 99 Bottles memorized? After all, it is a function (with a 3-element domain).