4 ms·

I mean, the cases where it tries to assign copyright to another person in a different year highlights that context other than the other text in the file is sema

by abeppu 5y ago

I mean, the cases where it tries to assign copyright to another person in a different year highlights that context other than the other text in the file is semantically extremely important, and not considered by this approach. Merely generating text which looks appropriate to the model given surrounding text is ... misguided? If you think about it, program synthesis is one of the few problems in which the system can have a perfectly faithful model dynamics of the problem domain. It can run any candidate it generates. It can examine the program graph. It can look at what parts of the environment were changed. To leave all that on the table in favor of blurting out text that seems to go with other text is like the toddler who knows that "five" comes after "four", but who cannot yet point to the pile of four candies. You gotta know the referents, not just the symbols. No one wants a half-broken Chinese Room.

whimsicalism 5y ago
> generating text which looks appropriate to the model given surrounding text is ... misguided? Agreed - it represents a failure to adequately model/understand the task, but I don't think it is a "fundamental weakness" of text-based 'Chinese room' approaches. > You gotta know the referents, not just the symbols. No one wants a half-broken Chinese Room. "Knowing the referents" is not at all clearly defined. It's totally possible that, under the constraint of optimizing for next-word prediction, the model could develop an understanding of what the referents are. You can't underestimate the level of complex behavior emerging from a big enough system under optimization. After all, all the crazy stuff we do - coding, art, etc. is produced by a system under evolutionary optimization pressure to make more of itself.
- abeppu 5y ago
  > "Knowing the referents" is not at all clearly defined. It's totally possible that, under the constraint of optimizing for next-word prediction, the model could develop an understanding of what the referents are. Well, in this case, it would have been good to understand that "V. Petkov" is a person unrelated to the project being written, and that "2015" is a year and not the one we're currently in. Sometimes the referent will be a method defined in an external library, which perhaps has a signature, and constraints about inputs, or properties which apply to return values. > You can't underestimate the level of complex behavior emerging from a big enough system under optimization. After all, all the crazy stuff we do - coding, art, etc. is produced by a system under evolutionary optimization pressure to make more of itself. I think this can verge into a kind of magical thinking. Yes, humans also look like neural nets, and we might even be optimizing for something. But we learn to program (and we do our best job programming) by having a goal for program behavior, and we use interactive access to try to run something, get an error, set a break point, try again, etc. I challenge anyone to try to learn to "code" by never being given any specific tasks, never interacting with docs about the language, an interpreter, a compiler, etc, but merely to try to fill in the blank in paper code snippets. You might learn to fill in some blanks. I highly doubt you would learn to code. This is totally a case where the textual representation of programs is easier to get and train against, and that tail is being allowed to wag the dog to frame both the problem and the product. None of this is to say that high-bandwidth DNN approaches don't have a place here -- but I think we should be looking at language-specific models where the DNN receives information about context (including some partial description of behavior) and outputs of the DNN are something like the weights in a PCFG that is used in the program search.

by abeppu 5y ago

whimsicalism 5y ago

> generating text which looks appropriate to the model given surrounding text is ... misguided? Agreed - it represents a failure to adequately model/understand the task, but I don't think it is a "fundamental weakness" of text-based 'Chinese room' approaches. > You gotta know the referents, not just the symbols. No one wants a half-broken Chinese Room. "Knowing the referents" is not at all clearly defined. It's totally possible that, under the constraint of optimizing for next-word prediction, the model could develop an understanding of what the referents are. You can't underestimate the level of complex behavior emerging from a big enough system under optimization. After all, all the crazy stuff we do - coding, art, etc. is produced by a system under evolutionary optimization pressure to make more of itself.

abeppu 5y ago
> "Knowing the referents" is not at all clearly defined. It's totally possible that, under the constraint of optimizing for next-word prediction, the model could develop an understanding of what the referents are. Well, in this case, it would have been good to understand that "V. Petkov" is a person unrelated to the project being written, and that "2015" is a year and not the one we're currently in. Sometimes the referent will be a method defined in an external library, which perhaps has a signature, and constraints about inputs, or properties which apply to return values. > You can't underestimate the level of complex behavior emerging from a big enough system under optimization. After all, all the crazy stuff we do - coding, art, etc. is produced by a system under evolutionary optimization pressure to make more of itself. I think this can verge into a kind of magical thinking. Yes, humans also look like neural nets, and we might even be optimizing for something. But we learn to program (and we do our best job programming) by having a goal for program behavior, and we use interactive access to try to run something, get an error, set a break point, try again, etc. I challenge anyone to try to learn to "code" by never being given any specific tasks, never interacting with docs about the language, an interpreter, a compiler, etc, but merely to try to fill in the blank in paper code snippets. You might learn to fill in some blanks. I highly doubt you would learn to code. This is totally a case where the textual representation of programs is easier to get and train against, and that tail is being allowed to wag the dog to frame both the problem and the product. None of this is to say that high-bandwidth DNN approaches don't have a place here -- but I think we should be looking at language-specific models where the DNN receives information about context (including some partial description of behavior) and outputs of the DNN are something like the weights in a PCFG that is used in the program search.