One of the genius things about advent of code is that the server never needs to touch unknown, potentially malicious, code. The server just compares the claimed

One of the genius things about advent of code is that the server never needs to touch unknown, potentially malicious, code. The server just compares the claimed answer to the stored answer. Maybe one way it could do the performance testing is: 24 hours after the puzzle is announced, a "speed test input" becomes available. The speed test input would be substantially larger than normal inputs. Because people have had time to write the code, all they have to do is dump the input into the code and put the output into the box the fastest. A notable bottleneck is that every problem does have to be solved by the server ahead of time, and if it starts taking ten minutes to solve each instance then that's expensive unless you use very small pools of inputs.