5 ms·

Experimentally Inventing a Programming Language Using Chat GPT

37 points by smallerfish 4y ago

aritmo 4y ago
This is very scary human-like. I was a teaching assistant and asked students to fix issues in their code. They acknowledged these issues but submitted again the same erroneous code.
mwaitjmp 4y ago
It’s very clever in how you can tell it that it was wrong and it admits this mistake. Has it been written in such a way that the operators instructions are given a higher weight compared to what it knows? I’ve asked it to generate python code and then format it with black. It reproduced the original code and then what was supposed to be the formatted code, but it was identical. When told of this it admitted it had made a mistake and this time correctly outputted the formatted code.
- kietay 4y ago
  The operator prompts becomes part of the input vector (it has a certain context depth it can accommodate), whereas the data it was trained on actually affects the models weights and biases. You can think of it like “evaluating this statement in light of the prior context against the training set”.
MattPalmer1086 4y ago
Pretty amazing interaction. Every time I've tried ChatGPT I get two questions in, an error happens and then all subsequent requests are rejected and I'm told to slow down, even if I wait quite a bit. Maybe it's just under high demand right now.
ilaksh 4y ago
I think it may need a more explicit description other than "immutable" for how they want the primes calculation to work.
hmsimha 4y ago
I actually got it to work through debugging its own code by running it and giving it error messages (which reference the line), or describing the thing that wasn't working. I didn't try asking it to add line numbers because I figured it has a tenuous understanding of the relationship between tokens as it is, but it seemed to understand what needed to be fixed from the error messages, the output (which was generated by log commands it added upon my prompting), and a description of the problem. See https://github.com/HartS/gpt-hacker-news-extension https://github.com/HartS/gpt-hacker-news-extension In the situations where it gave me partial output, I copied the last line and said "continue from "...""

37 points by smallerfish 4y ago

This is very scary human-like. I was a teaching assistant and asked students to fix issues in their code. They acknowledged these issues but submitted again the same erroneous code.

mwaitjmp 4y ago

It’s very clever in how you can tell it that it was wrong and it admits this mistake. Has it been written in such a way that the operators instructions are given a higher weight compared to what it knows? I’ve asked it to generate python code and then format it with black. It reproduced the original code and then what was supposed to be the formatted code, but it was identical. When told of this it admitted it had made a mistake and this time correctly outputted the formatted code.

kietay 4y ago
The operator prompts becomes part of the input vector (it has a certain context depth it can accommodate), whereas the data it was trained on actually affects the models weights and biases. You can think of it like “evaluating this statement in light of the prior context against the training set”.

MattPalmer1086 4y ago

Pretty amazing interaction. Every time I've tried ChatGPT I get two questions in, an error happens and then all subsequent requests are rejected and I'm told to slow down, even if I wait quite a bit. Maybe it's just under high demand right now.

ilaksh 4y ago

I think it may need a more explicit description other than "immutable" for how they want the primes calculation to work.

hmsimha 4y ago

I actually got it to work through debugging its own code by running it and giving it error messages (which reference the line), or describing the thing that wasn't working. I didn't try asking it to add line numbers because I figured it has a tenuous understanding of the relationship between tokens as it is, but it seemed to understand what needed to be fixed from the error messages, the output (which was generated by log commands it added upon my prompting), and a description of the problem. See https://github.com/HartS/gpt-hacker-news-extension https://github.com/HartS/gpt-hacker-news-extension In the situations where it gave me partial output, I copied the last line and said "continue from "...""