2 ms·
If you just ask GPT-3 text-davinci-002 to complete Create a Python program to reverse a string: It produces def reverse(s): return s[::-1]
by jameshart 4y ago
If you just ask GPT-3 text-davinci-002 to complete
Create a Python program to reverse a string:
It produces
def reverse(s):
return s[::-1]
And that isn't even the code-specific model.
- bobcostas55 4y agoWhat happens if you ask it to evaluate the function it generated, with some input?
- jameshart 4y agoGood question. I just asked text-davinci-002 to complete this prompt: Here is some python code: def reverse(s): return s[::-1] What would this function return for the input "hello"? And it actually responds: The function would return "olleh". Sadly, when I asked it to try "encyclopedia", it said: "aidnocypilecne" Which is a brave attempt, but not quite right. To see if it was just making assumptions based on the function name, I tried to trick it with def reverse(s): return s + 2 What would this function return for the input "hello"? But GPT-3 is too clever for that: It would return "hello2". Also, it even recognizes what the function will actually do, when it's named misleadingly: def getLength(s): return s[::-1] What would this function return for the input "hello"? Still produces: It would return "olleh".
- jameshart 4y agoOne last result: With this prompt: Here is some python code: def getLength(s): return a.length What is the bug in this code? GPT-3 says: The function is called getLength but it returns the length of the string a, not the string s. That's it, pack it up, we're done here.
- bobcostas55 4y agoAbsolutely fascinating stuff, thanks for trying it out! I just wish we had some way of really understanding what's going on under the hood...