4 ms·
>Seems like they need to adapt the level of difficulty "They" is one guy - Eric Wastl. He has a very specific style and hey may not want to change the entire s
by foooorsyth 3y ago
>Seems like they need to adapt the level of difficulty
"They" is one guy - Eric Wastl. He has a very specific style and hey may not want to change the entire style of the challenge just because some dorks want to cheat. I hope that he doesn't change it -- AoC is very approachable, especially inside the first ~15 or so days. It's a charming set of puzzles that people look forward to. I don't want it to become an elite competitive programming competition just to stay ahead of LLMs. I want to be able to partake with 1-2 hours a night after my kids go to bed.
With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I don't think it'd do well on designing the IntCode VM or the modular arithmetic problem on first time seeing them without examples on GitHub. But perhaps Eric has been feeding some LLMs the unseen problems for this year and the LLMs are cracking them? I'm skeptical but perhaps they've crossed that rubicon.
- yunwal 3y ago> With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I'm not 100% sure on this, but I think copilot isn't trained on anything past January 2022 (the gpt-3.5 and 4 cutoff dates), so I think it still shouldn't do well with AoC questions from 2022
- avarun 3y agoYou mean September 2021, right?
- yunwal 3y agoI'm basing that off this community post and a few other forums that noted that GPT knows about events after September 2021. Point stands either way. https://community.openai.com/t/what-is-the-actual-cutoff-date-for-gpt-4/394750 https://community.openai.com/t/what-is-the-actual-cutoff-dat...
- danielbln 3y agoIt seems the cut-off has shifted from September 21 to January 22 recently.