7 ms·

University of Waterloo withholds coding contest results over suspected AI use

94 points by amichail 2y ago

mkbelieve 2y ago
[flagged]
- anonymousab 2y ago
  It seems that these tests had a rule about not using generative AI from the start. So it would be more like someone entering a fingerpainting contest and using a paint brush. EDIT: Oh, I misread, that was the USACO competition which had the explicit rule.
  - Jtsummers 2y ago
    > All other Internet use is forbidden. This includes email, chat, web search, code forums, and generative AI such as CoPilot or ChatGPT. https://cccgrader.com/rules.pdf https://cccgrader.com/rules.pdf It was against the rules.
- jagger27 2y ago
  HN has such an impeccable knack to optimize towards the least human interpretation of any given topic. You reflect your idols well.
  - reneretord 2y ago
    [dead]
- Deukhoofd 2y ago
  It's far closer to "getting someone else to do your work for you" than it is to using tools. That's always been looked down on in competitions.
  - bslanej 2y ago
    Have you tried using code generated by an LLM? It rarely runs at all until you fix it yourself. It reminds me of how ezines would publish source codes to run exploits with slight errors so script kiddies couldn’t run them if they didn’t know how to fix them.
    - ToucanLoucan 2y ago
      > Have you tried using code generated by an LLM? It rarely runs at all until you fix it yourself. In College Humor's Brennan's exasperated tone: THEN WHAT ARE WE DOING!? Like Jesus fuck. I feel insane. We scraped the internet, broke tons of trust in our community, fed tons of code we didn't ask for into an industrial shredder, and worked out nonsense generators that can make awful code that barely, and apparently sometimes just doesn't work, burning enough electricity to power several small countries in the process and lighting billions of dollars on fire. What, and I can't stress this enough, the fuck are we doing anymore. I swear to God the entire valley needs to be pushed into the ocean and humanity will lurch forward 200 years.
      - bslanej 2y ago
        That’s hyperbolic. An LLM gets you like… 90% of the way there. That’s much better than the previous baseline of 0%
    - nneonneo 2y ago
      We’re talking about coding competitions here. These models have been fed the entirety of ACM, SPOJ, LeetCode, and whatever other competitions they can get their hands on. They are good at constrained coding competition tasks, and certainly strong enough to place highly in a competition meant for high school students.
    - Deukhoofd 2y ago
      To be entirely fair, that's the same experience I've had with some coworkers.
- titaphraz 2y ago
  No. The guy showed up in Bows & Arrows contest with M1 Abrams.
  - Legend2440 2y ago
    Bigger question: Why are universities running bow & arrow contests in a world full of M1 tanks?
    - add-sub-mul-div 2y ago
      The former is a measure of a real skill. I could prompt my sister about how to drive a car but they wouldn't let me send her on a driving test and then give me a license.
    - yjftsjthsd-h 2y ago
      Bow & Arrows are still in the literal Olympics; there is clearly appetite for seeing what humans can do with less powerful tools. Actually the fact that there are races on foot, bike, and car is probably an interesting analogy; I for one would be a fan of having a spread of contests from "you may use only pen and paper" to "literally any tool you can use is fair game".
- ribcage 2y ago
  [dead]
- timewizard 2y ago
  It's like entering an art contest using a collection of stencils created by other people and used without attribution or credit. So, really, they're screwing over more than two groups of people at the same time. I guess that is some kind of progress.
- ToucanLoucan 2y ago
  I am genuinely curious why people like this are still making this argument long past the point where anyone who is even slightly informed realizes it's garbage. This is some "the earth is flat" shit in 2025.
- deleted 2y ago
  [deleted]
- vunderba 2y ago
  No. The better analogy would be if you placed a hidden outline of the drawing in advance and traced over it during the "competition".
StefanBatory 2y ago
"The decision to cancel the release of this year’s results will weigh most on Grade 12 students who won’t get a chance to do it again next year, Shin said. But he didn’t think the onus was on the university. “It’s obviously a cheater’s fault, right? They’re the ones who cheated. They’re the one to break the rules. It’s their fault morally and logically.” ... I guess so, but how it is fault of someone who did not cheat? They are lumped in together, like it's their fault too. Not that I don't get that, but still.
- mannykannot 2y ago
  They are the victims.
- winwang 2y ago
  I agree, it sucks to be lumped in. What's your proposed solution?
  - criddell 2y ago
    Allow the use of AIs.
  - gonzo41 2y ago
    Hand written coding. The wheel comes full circle.
    - morkalork 2y ago
      Computers with no internet access and nothing but the compiler chain and text documentation installed. Coding like it's 1980s again!
      - PontifexMinimus 2y ago
        Which is in any case a better test of the core of coding, which is about figuring out algorithms and data structures, not writing glue code to connect various APIs together.
RainyDayTmrw 2y ago
Archived/paywall bypass: https://archive.is/ZEfI8 https://archive.is/ZEfI8
RainyDayTmrw 2y ago
Take this article with a grain of salt. Much of its argument quotes from TurnItIn, which was already an unethical scourge in earlier days, and already ruined many students' lives with traditional information retrieval, but more recently pivoted to ruining students' lives with "AI" instead[1]. [1]: https://www.washingtonpost.com/technology/2023/08/14/prove-false-positive-ai-detection-turnitin-gptzero/ https://www.washingtonpost.com/technology/2023/08/14/prove-f... (archived/paywall bypass) https://archive.is/BoVlG https://archive.is/BoVlG
vander_elst 2y ago
Why isn't it possible to run these contexts in a computer room with no access to the Internet? This plus a minimal amount of supervision would remove cheating? Am I missing something? Maybe schools don't have a computer room these days?
- tmyklebu 2y ago
  The second stage (on-site, ~20 competitors) is done like that. The first stage (this one, with ballpark 10000 competitors) is distributed and done in very heterogeneous environments, and it relies on local high school teachers taking time out of their day to set up whatever's needed, proctor the contest, and grade the results.
  - Wowfunhappy 2y ago
    So the problem is that the teachers were dishonest or not proctoring well, right? It doesn't sound to me like AI was the problem per se. Before LLMs you could have had participants using Stack Overflow to cheat.
    - handoflixue 2y ago
      > Before LLMs you could have had participants using Stack Overflow to cheat. "could have", yes, but mostly didn't until LLMs showed up. The problem is that LLMs lower the bar to cheating: using Stack Overflow still requires a pretty solid knowledge of the domain (and in a professional coding context, usually wouldn't even count as cheating). The LLMs also let you do this much faster.
- seanmcdirmid 2y ago
  You’d also have to make sure they were using clean computers without any local models installed, and coding models are pretty easy to run without high end GPUs. I don’t know if I would consider this modern coding using modern coding tool chains anymore though. It might still be a nice competition to run, but it is increasingly displaced from what it means to be a professional programmer (I’ve stopped caring about these competition wins on resumes a long time ago).
tmyklebu 2y ago
This really is quite sad. The first stage of the CCC is written in pretty heterogeneous conditions. There are several thousand competitors across the world. Students use a language of their choice and their work is graded by their teacher against a rubric sent out by the contest organisers. I recall that the teacher has to send the work to Waterloo, postmarked within a couple of days after the contest. Sometime in the summer, the University of Waterloo then flies out and puts up the top 15 or 20 students in that more informal contest to compete in what was the "CCC Stage 2" and is now the "Canadian Computing Olympiad." This second contest is done under more controlled conditions and its results largely determine which four students will represent Canada at the International Olympiad in Informatics. The first stage has always relied on teachers' honesty. The contest organisers mail contest packets to teachers in advance of the contest. Teachers have some time between contest day and when the results need to be mailed back to Waterloo. Students are comparing notes and discussing the contest immediately after the contest. Nonetheless, in my experience (more than 15 years ago), all or almost all of the students who make it to the second stage deserved to be there. I hope that continues to be true.
methuselah_in 2y ago
Well these kids who want to become coders and use ai even for competitions, i don't understand what they will achiev!

94 points by amichail 2y ago

mkbelieve 2y ago

[flagged]

anonymousab 2y ago
It seems that these tests had a rule about not using generative AI from the start. So it would be more like someone entering a fingerpainting contest and using a paint brush. EDIT: Oh, I misread, that was the USACO competition which had the explicit rule.
- Jtsummers 2y ago
  > All other Internet use is forbidden. This includes email, chat, web search, code forums, and generative AI such as CoPilot or ChatGPT. https://cccgrader.com/rules.pdf https://cccgrader.com/rules.pdf It was against the rules.
jagger27 2y ago
HN has such an impeccable knack to optimize towards the least human interpretation of any given topic. You reflect your idols well.
- reneretord 2y ago
  [dead]
Deukhoofd 2y ago
It's far closer to "getting someone else to do your work for you" than it is to using tools. That's always been looked down on in competitions.
- bslanej 2y ago
  Have you tried using code generated by an LLM? It rarely runs at all until you fix it yourself. It reminds me of how ezines would publish source codes to run exploits with slight errors so script kiddies couldn’t run them if they didn’t know how to fix them.
  - ToucanLoucan 2y ago
    > Have you tried using code generated by an LLM? It rarely runs at all until you fix it yourself. In College Humor's Brennan's exasperated tone: THEN WHAT ARE WE DOING!? Like Jesus fuck. I feel insane. We scraped the internet, broke tons of trust in our community, fed tons of code we didn't ask for into an industrial shredder, and worked out nonsense generators that can make awful code that barely, and apparently sometimes just doesn't work, burning enough electricity to power several small countries in the process and lighting billions of dollars on fire. What, and I can't stress this enough, the fuck are we doing anymore. I swear to God the entire valley needs to be pushed into the ocean and humanity will lurch forward 200 years.
    - bslanej 2y ago
      That’s hyperbolic. An LLM gets you like… 90% of the way there. That’s much better than the previous baseline of 0%
  - nneonneo 2y ago
    We’re talking about coding competitions here. These models have been fed the entirety of ACM, SPOJ, LeetCode, and whatever other competitions they can get their hands on. They are good at constrained coding competition tasks, and certainly strong enough to place highly in a competition meant for high school students.
  - Deukhoofd 2y ago
    To be entirely fair, that's the same experience I've had with some coworkers.
titaphraz 2y ago
No. The guy showed up in Bows & Arrows contest with M1 Abrams.
- Legend2440 2y ago
  Bigger question: Why are universities running bow & arrow contests in a world full of M1 tanks?
  - add-sub-mul-div 2y ago
    The former is a measure of a real skill. I could prompt my sister about how to drive a car but they wouldn't let me send her on a driving test and then give me a license.
  - yjftsjthsd-h 2y ago
    Bow & Arrows are still in the literal Olympics; there is clearly appetite for seeing what humans can do with less powerful tools. Actually the fact that there are races on foot, bike, and car is probably an interesting analogy; I for one would be a fan of having a spread of contests from "you may use only pen and paper" to "literally any tool you can use is fair game".
ribcage 2y ago
[dead]
timewizard 2y ago
It's like entering an art contest using a collection of stencils created by other people and used without attribution or credit. So, really, they're screwing over more than two groups of people at the same time. I guess that is some kind of progress.
ToucanLoucan 2y ago
I am genuinely curious why people like this are still making this argument long past the point where anyone who is even slightly informed realizes it's garbage. This is some "the earth is flat" shit in 2025.
deleted 2y ago
[deleted]
vunderba 2y ago
No. The better analogy would be if you placed a hidden outline of the drawing in advance and traced over it during the "competition".

StefanBatory 2y ago

"The decision to cancel the release of this year’s results will weigh most on Grade 12 students who won’t get a chance to do it again next year, Shin said. But he didn’t think the onus was on the university. “It’s obviously a cheater’s fault, right? They’re the ones who cheated. They’re the one to break the rules. It’s their fault morally and logically.” ... I guess so, but how it is fault of someone who did not cheat? They are lumped in together, like it's their fault too. Not that I don't get that, but still.

mannykannot 2y ago
They are the victims.
winwang 2y ago
I agree, it sucks to be lumped in. What's your proposed solution?
- criddell 2y ago
  Allow the use of AIs.
- gonzo41 2y ago
  Hand written coding. The wheel comes full circle.
  - morkalork 2y ago
    Computers with no internet access and nothing but the compiler chain and text documentation installed. Coding like it's 1980s again!
    - PontifexMinimus 2y ago
      Which is in any case a better test of the core of coding, which is about figuring out algorithms and data structures, not writing glue code to connect various APIs together.

RainyDayTmrw 2y ago

Archived/paywall bypass: https://archive.is/ZEfI8 https://archive.is/ZEfI8

RainyDayTmrw 2y ago

Take this article with a grain of salt. Much of its argument quotes from TurnItIn, which was already an unethical scourge in earlier days, and already ruined many students' lives with traditional information retrieval, but more recently pivoted to ruining students' lives with "AI" instead[1]. [1]: https://www.washingtonpost.com/technology/2023/08/14/prove-false-positive-ai-detection-turnitin-gptzero/ https://www.washingtonpost.com/technology/2023/08/14/prove-f... (archived/paywall bypass) https://archive.is/BoVlG https://archive.is/BoVlG

vander_elst 2y ago

Why isn't it possible to run these contexts in a computer room with no access to the Internet? This plus a minimal amount of supervision would remove cheating? Am I missing something? Maybe schools don't have a computer room these days?

tmyklebu 2y ago
The second stage (on-site, ~20 competitors) is done like that. The first stage (this one, with ballpark 10000 competitors) is distributed and done in very heterogeneous environments, and it relies on local high school teachers taking time out of their day to set up whatever's needed, proctor the contest, and grade the results.
- Wowfunhappy 2y ago
  So the problem is that the teachers were dishonest or not proctoring well, right? It doesn't sound to me like AI was the problem per se. Before LLMs you could have had participants using Stack Overflow to cheat.
  - handoflixue 2y ago
    > Before LLMs you could have had participants using Stack Overflow to cheat. "could have", yes, but mostly didn't until LLMs showed up. The problem is that LLMs lower the bar to cheating: using Stack Overflow still requires a pretty solid knowledge of the domain (and in a professional coding context, usually wouldn't even count as cheating). The LLMs also let you do this much faster.
seanmcdirmid 2y ago
You’d also have to make sure they were using clean computers without any local models installed, and coding models are pretty easy to run without high end GPUs. I don’t know if I would consider this modern coding using modern coding tool chains anymore though. It might still be a nice competition to run, but it is increasingly displaced from what it means to be a professional programmer (I’ve stopped caring about these competition wins on resumes a long time ago).

tmyklebu 2y ago

This really is quite sad. The first stage of the CCC is written in pretty heterogeneous conditions. There are several thousand competitors across the world. Students use a language of their choice and their work is graded by their teacher against a rubric sent out by the contest organisers. I recall that the teacher has to send the work to Waterloo, postmarked within a couple of days after the contest. Sometime in the summer, the University of Waterloo then flies out and puts up the top 15 or 20 students in that more informal contest to compete in what was the "CCC Stage 2" and is now the "Canadian Computing Olympiad." This second contest is done under more controlled conditions and its results largely determine which four students will represent Canada at the International Olympiad in Informatics. The first stage has always relied on teachers' honesty. The contest organisers mail contest packets to teachers in advance of the contest. Teachers have some time between contest day and when the results need to be mailed back to Waterloo. Students are comparing notes and discussing the contest immediately after the contest. Nonetheless, in my experience (more than 15 years ago), all or almost all of the students who make it to the second stage deserved to be there. I hope that continues to be true.

methuselah_in 2y ago

Well these kids who want to become coders and use ai even for competitions, i don't understand what they will achiev!