4 ms·

Its not executing code, its guessing what the answer is, the same way you would do mentally.

dakiol 4y ago
I know it's guessing it... but if by guessing it outputs the correct answer, well then what's the difference between the AI and the actual Python compiler? That's my question: how is the AI able to guess the correct answer just like the actual Python compiler? Outputting code and poems is one thing (there is no "correct" poem nor piece of code). Executing code is another: there is only one correct execution output for a given piece of code.
- nullish_signal 4y ago
  I would theorize that, the AI is "merely" manipulating an internal array (either the text-answer box, or some internal thinking layer) while abiding by the "compilation rules" of whatever Language the user has input. This producing a result that mirrors that of a proper compiler. To test this hypothesis, I would input python programs with more variables/functions/language features. I would expect the non-compiler AI to return an invalid response at some point, unable to "step theough" complex programs with only basic pattern matching. I don't know if rigorous CS proofs of "Turing Machine-Emulation" also hold for a Language model.
- carl_dr 4y ago
  I used the prompt ‘Calculate the permutations of the world “safer” in python’ It gave me the correct code (which shows all 120 permutations when ran in Python) : import itertools word = "safer" # Get all permutations of the word permutations = itertools.permutations(word) # Print each permutation for permutation in permutations: print(permutation) It then showed the output, but got it wrong. It listed 24 permutations, then said ‘As you can see, there are 24 possible permutations of the word "safer".‘ Interestingly, if you ask it how many permutations “safer” has, not asking about code, it invents some maths that produces 24 (see my reply in another post), so I guess it’s the understanding of permutations which is off.

I know it's guessing it... but if by guessing it outputs the correct answer, well then what's the difference between the AI and the actual Python compiler? That's my question: how is the AI able to guess the correct answer just like the actual Python compiler? Outputting code and poems is one thing (there is no "correct" poem nor piece of code). Executing code is another: there is only one correct execution output for a given piece of code.

nullish_signal 4y ago
I would theorize that, the AI is "merely" manipulating an internal array (either the text-answer box, or some internal thinking layer) while abiding by the "compilation rules" of whatever Language the user has input. This producing a result that mirrors that of a proper compiler. To test this hypothesis, I would input python programs with more variables/functions/language features. I would expect the non-compiler AI to return an invalid response at some point, unable to "step theough" complex programs with only basic pattern matching. I don't know if rigorous CS proofs of "Turing Machine-Emulation" also hold for a Language model.
carl_dr 4y ago
I used the prompt ‘Calculate the permutations of the world “safer” in python’ It gave me the correct code (which shows all 120 permutations when ran in Python) : import itertools word = "safer" # Get all permutations of the word permutations = itertools.permutations(word) # Print each permutation for permutation in permutations: print(permutation) It then showed the output, but got it wrong. It listed 24 permutations, then said ‘As you can see, there are 24 possible permutations of the word "safer".‘ Interestingly, if you ask it how many permutations “safer” has, not asking about code, it invents some maths that produces 24 (see my reply in another post), so I guess it’s the understanding of permutations which is off.