4 ms·

What happens if you ask it to evaluate the function it generated, with some input?

• jameshart 4y ago
  Good question. I just asked text-davinci-002 to complete this prompt: Here is some python code: def reverse(s): return s[::-1] What would this function return for the input "hello"? And it actually responds: The function would return "olleh". Sadly, when I asked it to try "encyclopedia", it said: "aidnocypilecne" Which is a brave attempt, but not quite right. To see if it was just making assumptions based on the function name, I tried to trick it with def reverse(s): return s + 2 What would this function return for the input "hello"? But GPT-3 is too clever for that: It would return "hello2". Also, it even recognizes what the function will actually do, when it's named misleadingly: def getLength(s): return s[::-1] What would this function return for the input "hello"? Still produces: It would return "olleh".
  • jameshart 4y ago
    One last result: With this prompt: Here is some python code: def getLength(s): return a.length What is the bug in this code? GPT-3 says: The function is called getLength but it returns the length of the string a, not the string s. That's it, pack it up, we're done here.
    • bobcostas55 4y ago
      Absolutely fascinating stuff, thanks for trying it out! I just wish we had some way of really understanding what's going on under the hood...