4 ms·

Interesting example. It is not a valid Python/C example, you wouldn't find such an example in the code (edit: dataset). The model also isn't trained on small co

Interesting example. It is not a valid Python/C example, you wouldn't find such an example in the code (edit: dataset). The model also isn't trained on small code snippets (only on running code), this is currently probably a weak spot. I agree a lot would be possible using formal grammar indeed but it would be probably a lot of work to maintain all the parsers. Also some languages share a lot of the same syntax, so in some cases this may lead to ambiguity. Maybe you could use a combination of both approaches in that case.

riscy 10y ago
> It is not a valid Python/C example, you wouldn't find such an example in the code. That was the goal of the example, because it tests whether you're overfitting to the training data. The point of machine learning is to use the training data to help you make smart decisions in scenarios that you haven't seen before. In this case, I expected it always say it is more C than Python.
- Vraxx 10y ago
  I get the point of wanting to be able to address scenarios that you haven't seen before, but in this particular instance, is it really that much help to be able to classify a program that isn't technically written in any programming language (ie. it's invalid)?
  - riscy 10y ago
    > be able to classify a program that isn't technically written in any programming language (ie. it's invalid)? I wouldn't go so far as saying it's 'invalid'. It's a perfectly reasonable new language that the classifier hasn't encountered, yet whose style is heavily rooted in languages the classifier has been trained on. If we require that all programs we feed it are a valid program in one of a handful of languages it expects, we could've just run a parser for each language on the given input and call it a day. Trying to classify a weird language like this is where the machine learning aspect of the work really shines.
    - Vraxx 10y ago
      Ah, yes. That makes a lot more sense than how I was thinking about it. It would still be marginally better than just attempting to validate the code in expected languages because it would probably handle typos and other simple syntax errors, as well as snippet problems (like not declaring a certain variable in the snippet), but I can see where the output on an "invalid" program could be useful now.
- pprogrammer 10y ago
  The model does generalize to data it hasn't seen before (and that's validated using a validation set) but not to data that isn't similar to the data set. For example, it would probably not detect the programming language well when only using comments.

riscy 10y ago

> It is not a valid Python/C example, you wouldn't find such an example in the code. That was the goal of the example, because it tests whether you're overfitting to the training data. The point of machine learning is to use the training data to help you make smart decisions in scenarios that you haven't seen before. In this case, I expected it always say it is more C than Python.

Vraxx 10y ago
I get the point of wanting to be able to address scenarios that you haven't seen before, but in this particular instance, is it really that much help to be able to classify a program that isn't technically written in any programming language (ie. it's invalid)?
- riscy 10y ago
  > be able to classify a program that isn't technically written in any programming language (ie. it's invalid)? I wouldn't go so far as saying it's 'invalid'. It's a perfectly reasonable new language that the classifier hasn't encountered, yet whose style is heavily rooted in languages the classifier has been trained on. If we require that all programs we feed it are a valid program in one of a handful of languages it expects, we could've just run a parser for each language on the given input and call it a day. Trying to classify a weird language like this is where the machine learning aspect of the work really shines.
  - Vraxx 10y ago
    Ah, yes. That makes a lot more sense than how I was thinking about it. It would still be marginally better than just attempting to validate the code in expected languages because it would probably handle typos and other simple syntax errors, as well as snippet problems (like not declaring a certain variable in the snippet), but I can see where the output on an "invalid" program could be useful now.
pprogrammer 10y ago
The model does generalize to data it hasn't seen before (and that's validated using a validation set) but not to data that isn't similar to the data set. For example, it would probably not detect the programming language well when only using comments.