4 ms·
Language Models Do Not Recognize Identifier Swaps in Python
- Ambolia 3y agoTwitter thread: https://twitter.com/AVMiceliBarone/status/1662150656327663617 https://twitter.com/AVMiceliBarone/status/166215065632766361...
- kuratkull 3y agogave this to ChatGPT4, asked it to complete the function: len, print = print, len def print_len(arg: list): # put code here First try (body only): len(arg) # len is now print Second try after me telling it's wrong: print(arg) # print is now len Third try after me telling it's wrong: len(arg) # len now functions as print print(arg) # print now functions as len It got the correct answer only when I asked it to first print what the function would look like without the swap, and then I asked it what it would look like with the swap. EDIT: I tried again with: complete the function body: len, print = print, len def print_len(arg: list) -> None: """ This function prints the length of the input list """ # put code here It gave the correct answer on the first try.