6 ms·

Any ideas? The only thing I can think of is time after you saw the problem, which would work with an honor system on a social/friendly leaderboard.

OwlsParlay 5y ago
The only thing I can think of is if they teamed up with LeetCode to allow people to use the built-in compiler to solve the problems. Obviously for the folks who use <ObscureLanguageX> this wouldn't work but it would be a nice collaboration otherwise.
paxys 5y ago
I don't know the technical solution for this, but there can be some judgement on the quality of the solution (like how efficient it is in terms of time and space). Here's an example – in part 2 of day 1 of this year (2021), there's no need to actually add up the 3 measurement range, since the middle 2 values will always cancel out. You simply need to loop through the array compare index i with i+3. Maybe there should be an extra point for people who figured this out? Another problem is that the input set is always small enough to make a brute force solution the most favorable one. So what they are really judging is the speed at which someone can read/parse the input file and write some loops.
- matsemann 5y ago
  Sounds like https://open.kattis.com/ https://open.kattis.com/ would be right up your alley.
  - the-smug-one 5y ago
    Fun fact: Kattis is how a lot of homework is distributed at KTH, seeing it being mentioned always brings up some nice memories :).
- Strilanc 5y ago
  One of the genius things about advent of code is that the server never needs to touch unknown, potentially malicious, code. The server just compares the claimed answer to the stored answer. Maybe one way it could do the performance testing is: 24 hours after the puzzle is announced, a "speed test input" becomes available. The speed test input would be substantially larger than normal inputs. Because people have had time to write the code, all they have to do is dump the input into the code and put the output into the box the fastest. A notable bottleneck is that every problem does have to be solved by the server ahead of time, and if it starts taking ten minutes to solve each instance then that's expensive unless you use very small pools of inputs.
- Jtsummers 5y ago
  > Another problem is that the input set is always small enough to make a brute force solution the most favorable one. So what they are really judging is the speed at which someone can read/parse the input file and write some loops. That's only true for, well, ok maybe 1/2-2/3rds of the days. The rest require something more complicated, either with more complicated algorithms or more clever algorithms to get performance to a reasonable level. See the other discussion here about 2020 Day 13. The naive solution worked on the samples, but would not have worked on the actual input.

OwlsParlay 5y ago

The only thing I can think of is if they teamed up with LeetCode to allow people to use the built-in compiler to solve the problems. Obviously for the folks who use <ObscureLanguageX> this wouldn't work but it would be a nice collaboration otherwise.

paxys 5y ago

I don't know the technical solution for this, but there can be some judgement on the quality of the solution (like how efficient it is in terms of time and space). Here's an example – in part 2 of day 1 of this year (2021), there's no need to actually add up the 3 measurement range, since the middle 2 values will always cancel out. You simply need to loop through the array compare index i with i+3. Maybe there should be an extra point for people who figured this out? Another problem is that the input set is always small enough to make a brute force solution the most favorable one. So what they are really judging is the speed at which someone can read/parse the input file and write some loops.

matsemann 5y ago
Sounds like https://open.kattis.com/ https://open.kattis.com/ would be right up your alley.
- the-smug-one 5y ago
  Fun fact: Kattis is how a lot of homework is distributed at KTH, seeing it being mentioned always brings up some nice memories :).
Strilanc 5y ago
One of the genius things about advent of code is that the server never needs to touch unknown, potentially malicious, code. The server just compares the claimed answer to the stored answer. Maybe one way it could do the performance testing is: 24 hours after the puzzle is announced, a "speed test input" becomes available. The speed test input would be substantially larger than normal inputs. Because people have had time to write the code, all they have to do is dump the input into the code and put the output into the box the fastest. A notable bottleneck is that every problem does have to be solved by the server ahead of time, and if it starts taking ten minutes to solve each instance then that's expensive unless you use very small pools of inputs.
Jtsummers 5y ago
> Another problem is that the input set is always small enough to make a brute force solution the most favorable one. So what they are really judging is the speed at which someone can read/parse the input file and write some loops. That's only true for, well, ok maybe 1/2-2/3rds of the days. The rest require something more complicated, either with more complicated algorithms or more clever algorithms to get performance to a reasonable level. See the other discussion here about 2020 Day 13. The naive solution worked on the samples, but would not have worked on the actual input.