6 ms·

I placed 1st in Advent Of Code Part 1 today, by having GPT-3 write the code. For those who are interested, here's the code I ran: https://github.com/max-sixty/

I placed 1st in Advent Of Code Part 1 today, by having GPT-3 write the code. For those who are interested, here's the code I ran: https://github.com/max-sixty/aoc-gpt https://github.com/max-sixty/aoc-gpt

guiambros 4y ago
Yeah, quite impressive. I did the same to test [1] (but after I had submitted my own solution), and it's incredible that it was able to solve it flawlessly. This is like AlphaGo beating the best human Go player a decade earlier than expected. ChatGPT looks truly magical, and it's just an interim beta version. Imagine when GPT4 is out. [1] https://twitter.com/GuiAmbros/status/1599282083838296064 https://twitter.com/GuiAmbros/status/1599282083838296064
- deleted 4y ago
  [deleted]
- bcrosby95 4y ago
  Interesting. I tried it and it had all sorts of problems, even in Python. It didn't split on the comma. It assumed all numbers would be 1 digit. When I tried to get it to fix those, it ran into other problems, and I decided to abandon the pursuit.
  - genidoi 4y ago
    I noticed that the specific starting prompt that GPT VM[0] author designed seems to be something like a local minimum in achieving stability of a pretend Linux. I replaced “Linux” in the prompt with “an Ubuntu instance in us-East-1 SSH’d into a Debian instance in us-East-2 ssh’d into a raspberry pi in germany……..” and at 4-5 levels deep it would forget the previous link when closing the “ssh connection” and go back to the first instance. I also thought the sentence towards the end of the prompt about “when I want to tell you something in English I’ll do it like {this}” was redundant, but removing it leads to some weirdness like “cat file.txt” printing out the man page for cat 75% of the time. md5sum / sha256sum also never work if the file contents are remotely non-trivial. so that seems to be the ceiling.. at least for this prompt :p [0] https://news.ycombinator.com/item?id=33847479 https://news.ycombinator.com/item?id=33847479
- LoganDark 4y ago
  It's really too bad that consumers won't see versions of these models for a really long time. OpenAI is so stingy with their models. It's for a good reason, but damn do I hate all the hoops they make you jump through for something so restricted. No, I don't have a phone number. Guess I won't be able to do anything. Time to go back to NovelAI.
awesomelvin 4y ago
You should really think about the ethics of using this. You could wait a few minutes before executing your script. Some people work really hard to get on the leaderboard and players like you destroy the concept.
- colordrops 4y ago
  What's the concept that this player destroyed?
- charcircuit 4y ago
  People who place high on the leaderboard already use prewritten libraries specialized for the kinds of problems advent of code asks. Calling out to gpt3 is not much different.
  - surfsvammel 4y ago
    I am not so sure. The ones that I am following, and watching, does not have very elaborate libraries.
- forty 4y ago
  My understanding is that the ranking of this game is more tied to the timezone you live in (since the ranking depends on the hour when you solve the problem), than anything else. Also it sounds like it is more targeted to people who want to have fun solving the problems than to people who want to compete for ranking. And last, until their rules say otherwise, I'd say using AI to solve the problems is perfectly fair.
  - nindalf 4y ago
    It's a game that can be played both casually and competitively, like many other games. But using artificial aids would be frowned upon even in casual contexts. I would be annoyed if my opponent in chess got suggestions from a chess AI or if my opponent in an online shooter was using an aimbot. I play both of these for fun, but a big part of the fun is competing with people of similar skill. I've never placed in the top 100 of AOC and I never will. But the folks who attempt it would be rightly miffed that someone cheated to get to the top. And please, don't talk about the effort needed to set up the cheating device. It took a lot of effort to (allegedly) cheat in chess with anal beads, but no one is praising that person.
fulafel 4y ago
Is there a record of the answer attempts it made? How many tries did it make?
- maximilianroos 4y ago
  No, unfortunately I didn't keep the logs. Very open to contributions to the code to do that!

guiambros 4y ago

Yeah, quite impressive. I did the same to test [1] (but after I had submitted my own solution), and it's incredible that it was able to solve it flawlessly. This is like AlphaGo beating the best human Go player a decade earlier than expected. ChatGPT looks truly magical, and it's just an interim beta version. Imagine when GPT4 is out. [1] https://twitter.com/GuiAmbros/status/1599282083838296064 https://twitter.com/GuiAmbros/status/1599282083838296064

deleted 4y ago
[deleted]
bcrosby95 4y ago
Interesting. I tried it and it had all sorts of problems, even in Python. It didn't split on the comma. It assumed all numbers would be 1 digit. When I tried to get it to fix those, it ran into other problems, and I decided to abandon the pursuit.
- genidoi 4y ago
  I noticed that the specific starting prompt that GPT VM[0] author designed seems to be something like a local minimum in achieving stability of a pretend Linux. I replaced “Linux” in the prompt with “an Ubuntu instance in us-East-1 SSH’d into a Debian instance in us-East-2 ssh’d into a raspberry pi in germany……..” and at 4-5 levels deep it would forget the previous link when closing the “ssh connection” and go back to the first instance. I also thought the sentence towards the end of the prompt about “when I want to tell you something in English I’ll do it like {this}” was redundant, but removing it leads to some weirdness like “cat file.txt” printing out the man page for cat 75% of the time. md5sum / sha256sum also never work if the file contents are remotely non-trivial. so that seems to be the ceiling.. at least for this prompt :p [0] https://news.ycombinator.com/item?id=33847479 https://news.ycombinator.com/item?id=33847479
LoganDark 4y ago
It's really too bad that consumers won't see versions of these models for a really long time. OpenAI is so stingy with their models. It's for a good reason, but damn do I hate all the hoops they make you jump through for something so restricted. No, I don't have a phone number. Guess I won't be able to do anything. Time to go back to NovelAI.

awesomelvin 4y ago

You should really think about the ethics of using this. You could wait a few minutes before executing your script. Some people work really hard to get on the leaderboard and players like you destroy the concept.

colordrops 4y ago
What's the concept that this player destroyed?
charcircuit 4y ago
People who place high on the leaderboard already use prewritten libraries specialized for the kinds of problems advent of code asks. Calling out to gpt3 is not much different.
- surfsvammel 4y ago
  I am not so sure. The ones that I am following, and watching, does not have very elaborate libraries.
forty 4y ago
My understanding is that the ranking of this game is more tied to the timezone you live in (since the ranking depends on the hour when you solve the problem), than anything else. Also it sounds like it is more targeted to people who want to have fun solving the problems than to people who want to compete for ranking. And last, until their rules say otherwise, I'd say using AI to solve the problems is perfectly fair.
- nindalf 4y ago
  It's a game that can be played both casually and competitively, like many other games. But using artificial aids would be frowned upon even in casual contexts. I would be annoyed if my opponent in chess got suggestions from a chess AI or if my opponent in an online shooter was using an aimbot. I play both of these for fun, but a big part of the fun is competing with people of similar skill. I've never placed in the top 100 of AOC and I never will. But the folks who attempt it would be rightly miffed that someone cheated to get to the top. And please, don't talk about the effort needed to set up the cheating device. It took a lot of effort to (allegedly) cheat in chess with anal beads, but no one is praising that person.

fulafel 4y ago

Is there a record of the answer attempts it made? How many tries did it make?

maximilianroos 4y ago
No, unfortunately I didn't keep the logs. Very open to contributions to the code to do that!