4 ms·

I've only tried it once, the other day. I asked it how I could improve my AoC day 4 Python code. It responsed with three points and potential modified code.

I've only tried it once, the other day. I asked it how I could improve my AoC day 4 Python code. It responsed with three points and potential modified code. Its understanding of the code I had written was impressive - it noticed I was looking for overlaps between two sets of data, and it made three suggestions for improvement. All of them looked great, but didn't work when tested. One worked when I changed it (having been pointed in the right direction), the other two just didn't work better than what I'd written, and were moot in terms of improvement. However, what I did notice was its comments to go along with the code were authoritative in tone, and I felt like I'd been schooled by a grand master. But that wasn't the case - it had a hit with one and a miss with the other two. I got the feeling it was like a certain kind of salesman; loud, confident, but in this case wrong. Good job I checked the code it came up with, otherwise I would have felt pretty bad.

dougmwne 4y ago
This is very fair and I have similar feelings about the accuracy and false confidence. Looking at your experience, the outcome is still impressive. You got 3 new solutions that you were able to get working with some tweaking and one of them was an improvement. Sounds like working with an experienced teammate who also makes mistakes but improves your code in the end with a new perspective and insight. Like… damn, this is all next word text prediction?
- pritambaral 4y ago
  > You got 3 new solutions that you were able to get working with some tweaking and one of them was an improvement. >> All of them looked great, but didn't work when tested. One worked when I changed it (having been pointed in the right direction), the other two just didn't work better than what I'd written, and were moot in terms of improvement.
shadowofneptune 4y ago
This behavior has been called 'confabulation,' and I think the name fits well. Other people call it 'hallucination.' Confabulation in people occurs after brain damage or dementia. They make stuff up while being convinced it was a memory of theirs. I haven't seen a model yet that rejects unlikely matches when it runs out of good material, it could make it more useful.

dougmwne 4y ago

This is very fair and I have similar feelings about the accuracy and false confidence. Looking at your experience, the outcome is still impressive. You got 3 new solutions that you were able to get working with some tweaking and one of them was an improvement. Sounds like working with an experienced teammate who also makes mistakes but improves your code in the end with a new perspective and insight. Like… damn, this is all next word text prediction?

pritambaral 4y ago
> You got 3 new solutions that you were able to get working with some tweaking and one of them was an improvement. >> All of them looked great, but didn't work when tested. One worked when I changed it (having been pointed in the right direction), the other two just didn't work better than what I'd written, and were moot in terms of improvement.

shadowofneptune 4y ago

This behavior has been called 'confabulation,' and I think the name fits well. Other people call it 'hallucination.' Confabulation in people occurs after brain damage or dementia. They make stuff up while being convinced it was a memory of theirs. I haven't seen a model yet that rejects unlikely matches when it runs out of good material, it could make it more useful.