3 ms·

Why isn't it possible to run these contexts in a computer room with no access to the Internet? This plus a minimal amount of supervision would remove cheating?

Why isn't it possible to run these contexts in a computer room with no access to the Internet? This plus a minimal amount of supervision would remove cheating? Am I missing something? Maybe schools don't have a computer room these days?

tmyklebu 1y ago
The second stage (on-site, ~20 competitors) is done like that. The first stage (this one, with ballpark 10000 competitors) is distributed and done in very heterogeneous environments, and it relies on local high school teachers taking time out of their day to set up whatever's needed, proctor the contest, and grade the results.
- Wowfunhappy 1y ago
  So the problem is that the teachers were dishonest or not proctoring well, right? It doesn't sound to me like AI was the problem per se. Before LLMs you could have had participants using Stack Overflow to cheat.
  - handoflixue 1y ago
    > Before LLMs you could have had participants using Stack Overflow to cheat. "could have", yes, but mostly didn't until LLMs showed up. The problem is that LLMs lower the bar to cheating: using Stack Overflow still requires a pretty solid knowledge of the domain (and in a professional coding context, usually wouldn't even count as cheating). The LLMs also let you do this much faster.
seanmcdirmid 1y ago
You’d also have to make sure they were using clean computers without any local models installed, and coding models are pretty easy to run without high end GPUs. I don’t know if I would consider this modern coding using modern coding tool chains anymore though. It might still be a nice competition to run, but it is increasingly displaced from what it means to be a professional programmer (I’ve stopped caring about these competition wins on resumes a long time ago).
- lovecg 1y ago
  > I’ve stopped caring about these competition wins on resumes a long time ago Some of the best engineers (I mean that in the broad sense, not just “were good at coding up algorithms”) I had the privilege of working alongside with scored highly on these competitions.
  - seanmcdirmid 1y ago
    I’m sure that there were other indicators on their CV that would allow you to filter them into an interview beyond their competition wins. They are just so pervasive on CVs (especially for people who went to school overseas) that they’ve just become noise to me.
adverbly 1y ago
Or maybe have cameras set up recording the test taking with some AI analysis to detect cheating.
- mikrl 1y ago
  >Using AI to catch AI cheaters Outside of being highly ironic, this is how you get a GAN arms race :)

tmyklebu 1y ago

The second stage (on-site, ~20 competitors) is done like that. The first stage (this one, with ballpark 10000 competitors) is distributed and done in very heterogeneous environments, and it relies on local high school teachers taking time out of their day to set up whatever's needed, proctor the contest, and grade the results.

Wowfunhappy 1y ago
So the problem is that the teachers were dishonest or not proctoring well, right? It doesn't sound to me like AI was the problem per se. Before LLMs you could have had participants using Stack Overflow to cheat.
- handoflixue 1y ago
  > Before LLMs you could have had participants using Stack Overflow to cheat. "could have", yes, but mostly didn't until LLMs showed up. The problem is that LLMs lower the bar to cheating: using Stack Overflow still requires a pretty solid knowledge of the domain (and in a professional coding context, usually wouldn't even count as cheating). The LLMs also let you do this much faster.

seanmcdirmid 1y ago

You’d also have to make sure they were using clean computers without any local models installed, and coding models are pretty easy to run without high end GPUs. I don’t know if I would consider this modern coding using modern coding tool chains anymore though. It might still be a nice competition to run, but it is increasingly displaced from what it means to be a professional programmer (I’ve stopped caring about these competition wins on resumes a long time ago).

lovecg 1y ago
> I’ve stopped caring about these competition wins on resumes a long time ago Some of the best engineers (I mean that in the broad sense, not just “were good at coding up algorithms”) I had the privilege of working alongside with scored highly on these competitions.
- seanmcdirmid 1y ago
  I’m sure that there were other indicators on their CV that would allow you to filter them into an interview beyond their competition wins. They are just so pervasive on CVs (especially for people who went to school overseas) that they’ve just become noise to me.

adverbly 1y ago

Or maybe have cameras set up recording the test taking with some AI analysis to detect cheating.

mikrl 1y ago
>Using AI to catch AI cheaters Outside of being highly ironic, this is how you get a GAN arms race :)