7 ms·
Evidence of inconsistencies in evaluation process and selection of winners
- deleted 3mo ago[deleted]
- onesandofgrain 3mo ago[flagged]
- reactordev 3mo agoAI is extremely useful, we just haven’t zeroed in on your specific use cases yet. Robotics has been transformed by it, IT and tech has been transformed by it. Finance and Legal have been transformed by it. To say it’s 95% useless is a personal bias. To me it’s 65% useful. As it can run in the background doing “chores” while I sleep.
- ndbe 3mo ago[dead]
- moron4hire 3mo agoApparently "being transformed" has "been transformed" because most things look a hell of a lot like the same thing these days.
- jappgar 3mo agoPeople's workday has been transformed by it. But I fail to see actual transformation beyond "more crap, faster." AI hasn't done anything we couldn't already do. It's just doing it faster and with more mistakes.
- jimbokun 3mo agoThat just isn’t true. AI is capable of performing a lot of grunt work reliably. Still must be reviewed. But a big productivity gain over doing everything yourself.
- glimshe 3mo agoWhile I agree with you in principle, I think the parent has a point here: where's the amazing product that couldn't have been done without AI? By now we should have seen some major new invention/company, incredibly fast revolutionary feature rollouts etc but I'm just seeing more of the same.
- reactordev 3mo agonot everything needs a unicorn product to validate it's existence. even just moving the field forward is sometimes enough to clear the air. what is clear is that we aren't going back.
- handoflixue 3mo agoCurious, what revolutionary products would you cite, that existed within 4 years of the underlying technology? I feel like within that time frame, you're looking at "AOL chatrooms" for the internet, and I'm not even sure what the killer app was for smart phones - GPS was around before that. Computers do admittedly claim the Apollo program pretty early in, so I'll definitely concede that one - just curious if other revolutions have really lived up to that bar. As someone who grew up on modems, I struggle to view the modern world as really having a "killer app" that really couldn't have been done in the 90s
- glimshe 3mo agoVisicalc was released 2 years after the Apple II was released. Visicalc changed the world for the better faster than AI has up to this point.
- reactordev 3mo agoAnthropic released Claude in March 2023. Claude Code was released in February 2025. I'd say Claude Code had about as much impact as Visicalc.
- 3mo ago
- fn-mote 3mo ago> AI hasn't done anything we couldn't already do. It's just doing it faster and with more mistakes. You forgot CHEAPER (at least now, burning VC money), which is a major motivating factor.
- risyachka 3mo agoWhen you factor in all the factors (fixing issues, implementing features that were added only because it was easy and shouldn’t exist in the first place, losing skills in the process (this one is a fact), losing grip on the codebase etc - it is not cheaper at all, probably more expensive
- moron4hire 3mo agoThis is offshoring all over again.
- prox 3mo agoYour perspective is a short arc. “Look what I can do now and look it made me way more productive.” I have no doubt it is true, you are on the up right now. However thinking of the long arc is important to, even though it has no consequence for you right now. AI is a force multiplayer and scarily dangerous in the wrong hands. We can already see by these discussions how uncertain things are. Just food for thought.
- cobbzilla 3mo agoYes, if the Cro-Magnons had guns they’d say “wow, hunting meat is WAY easier” and then experience massive-scale death in the future. But that kinda happened anyway in various places, just using much more primitive tools. Humans gonna human.
- fn-mote 3mo ago> AI is a force multiplayer and scarily dangerous in the wrong hands You have to spell this out a lot more if you want to have credibility. I’m not seeing anything in discussed here that seems scary.
- handoflixue 3mo agoAlready real: Automated scams & deepfake pornography. You can deepfake a real time video call to look and sound like someone else. If you're used to entering passwords this is easy to solve, but it's already a billion dollar industry. Moving into the future slightly: they're already getting decent at video games, and if they can win at Counterstrike they can probably also win at real-life Drone Warfare. I'd say the risks are mostly still hypothetical, though. There's a ton of reading out there if you take that idea seriously, but I don't blame you if you dismiss it as being a bit too "Science Fiction"
- prox 3mo agoI think the sibling comment shares some good examples. Maybe future drone armies is another one. That you don’t see anything scary that’s your prerogative of course.
- reactordev 3mo ago
- nsagent 3mo ago> As it can run in the background doing “chores” while I sleep. Slightly off topic, but this reminds me of the night family in Rick and Morty.
- datadrivenangel 3mo agoAI is extremely useful and 95% USELESS at the same time. LLM based chat AI systems are a poor version of all the things that are AI.
- alvah 3mo agoI think Ed Zitron has this market covered already
- cbg0 3mo agobeep-boop, [citation needed] on that "95%".
- deleted 3mo ago[deleted]
- lioeters 3mo agoBeep boop. Newly proposed international law mandates all AI-generated content to start with: "beep boop".
- Lomlioto 3mo agoSomeone might want to downvote you because you just state something which is very controversal and you do not add any arguments to your 'empty' comment. Its hard to even have a discussion because someone else needs to give you enough content like ask you first why do you even think that. So how do you define AI? LLMs? GenAI stuff? What is 95%? Does it mean that these 5% are unable to disrupt industries or does it mean for you that these 5% will change the world as we know it but stil 95% of other AI stuff is useless? I personally think that AI/AGI progress is faster than i expected it, I think its very useful already today, I also think we still need to build a lot of obvious stuff (like proper AI Agentic Platforms), more hardware, cheaper hardware etc. but the way quite clear, but some peple might think the current state is the AGI future people talk about it but I think we will only see this in 4/5-15 years and then it will have disrupted a lot.
- oguzkaanmavice 3mo ago[flagged]
- baal80spam 3mo ago> AI is 95% useless This is a ridiculous take. FWIW, I am NOT a bot. (beep-boop)
- subwaydash 3mo ago[dead]
- jimbokun 3mo agoAI has profound weaknesses, but is still extremely useful.
- simonw 3mo ago2024-era take.
- echelon 3mo ago[flagged]
- irasigman 3mo agoIt’s a shame that Arvix (and once thoughtful places like Kaggle) are used for self-promotion. I get people want to work at an AI lab but slopping it in public in this manner is counterproductive to the original intended purpose of these places.
- charcircuit 3mo agoHasn't this always been the case? Arxiv being used for self promotion and Kaggle being used to pivot into the industry. It is not a recent phenomenon.
- deleted 3mo ago[deleted]
- bonoboTP 3mo agoAcademia is itself self promotion. Conferences, publications, talks, all of it.
- a123b456c 3mo agoI understand why this external view exists, as dissemination is inherently part of the scientific mission; but if you look more carefully, you will see that it is simultaneously science promotion, and many of the best participants do not shamelessly promote themselves.
- bonoboTP 3mo agoI did a PhD, so not external. Sure one doesn't have to be maximally cynical but a lot of it is self promotion. And in context: the same can be said about Kaggle, about Youtubers, about music creators etc. Every endeavor is a mix of "pure" promotion of the art and of shameless self promotion and status games. The common factor is humans. My point was, academia is not more pure than the Kaggle guys who fish for industry jobs.
- probably_wrong 3mo ago
- jgilias 3mo agoIt was probably scored by AI too. Same reason why slop-filled resumes apparently work better these days.
- onesandofgrain 3mo agoAI reviews, AI approves, AI recruits
- mapt 3mo agoWe need a sufficiently advanced world model to ground-truth our large language models. EDIT: This was mostly satirical.
- onesandofgrain 3mo agoYes, let's fund some more trillions for that instead of curing cancer or making extremely effective solar panels or curing alzheimer and artherosclerosis. We could have spent these 10 trillion on so many better things.
- gchamonlive 3mo ago> Same reason why apparently slop-filled resumes work better these days. It'll also filter the kinds of employers that'll hire such candidates, so people that do this will likely land in terrible workplaces.
- throwfaraway135 3mo agoAI submissions and AI judges a match made in (AI) heaven.
- kasumispencer2 3mo agoA Slavoj Žižek-style "sticking a dildo into a fleshlight and have them do the sex for us" kind of situation, really
- kasumispencer2 3mo agoWelp, considering that one quick downvote, I suppose no one knows humor anymore... Edit: changing your downvote to something else won't change that fact you know
- The_Blade 3mo agoi'll throw on some Steely Dan to that, but if AI is a body without organs how can it do the sex for us
- anon7725 3mo agoSee also: AI PR authors & AI PR reviewers
- freedomben 3mo agoI would have agreed before seeing Co-Pilot (I was extremely skeptical about its usefulness), but after seeing results, I was wrong. It's actually pretty damn good at code reviewing PRs even when the PR was made entirely by AI. It doesn't seem like it should work, but it does
- _se 3mo agoNo... no it doesn't.
- hydevito 3mo agoadding "AI slop resumes and AI slop HR screening"
- biosboiii 3mo agoLLM-as-a-judge? Given that LLMs are trained with RL && LLM-as-a-judge, is it really cheating if real competitions use the same? Maybe the real alignment is the slop we decoded along the way
- ndbe 3mo agoThe amount of slop in the replies is just sad.
- subwaydash 3mo ago[dead]
- echelon 3mo agoEdit: title of the article originally contained the word "slop" and was negative / whiny in tone, but it has now been editorialized to a more neutral title. I'm being downvoted without that context. --- I'm sick of the word "slop". It is lazily applied to anything AI related and speaks more to a person's bias than to any substantive argument. I wish I could nuke every comment with that word from my feed.
- cliglot 3mo ago[flagged]
- inigyou 3mo agoHuman. They call it "my human"
- echelon 3mo ago[flagged]
- krapp 3mo ago[flagged]
- tomhow 3mo agoWe appreciate your contributions here, but can you dial back this kind of sentiment in your comments? > Go build and never speak to a human again, if that's what you want. Reject the humanity you despise It's a really unfair stereotype to apply to those you're disagreeing with, and I see you do it repeatedly here; characterizing those you disagree with, or the HN community in general, with terms like anti-humanity and anti-human. Yes I know there's a transhumanist element and an elitist element in Silicon Valley and among AI zealots. But it's not true of everyone or even many people who have big hopes for AI, and it's not a dominant sentiment on HN, or within YC. Most people I see being hopeful about AI are hopeful that it can make things better for humans – better jobs (more pleasant and better paying in real terms), better health/medicine and education that's more accessible to all, etc. Sure, it's not a given that this will eventuate, and there are plenty of unknowns and plenty of ways in which things could get worse if the wrong decisions are made. Which is exactly why we need to have ongoing, vigorous discussion, which is what we're always trying to cultivate here on HN. It's fine to express concerns about the real challenges and negative aspects of AI (and tech in general) and the potential for things to get worse. It's good to have that perspective represented. But, continuing to name or characterize your debating opponents, or the HN/YC/tech community in general as "anti-human" is unfair often-inaccurate. It's a low-substance slur that serves to poison discussions. Enough, please.
- nsagent 3mo agoSadly, the major ML/AI/NLP conferences are being inundated with AI slop papers. That will arguably have a bigger impact on the quality of research moving forward.
- Painsawman123 3mo ago" impact on the quality of research moving forward. " It'll affect everything that depends on manipulating symbols! The enormous body of knowledge humanity has accumulated over the past 6.000 years or so is about to be flooded with slop!And That's the real threat genai poses to humans that i don't see anyone talking about..
- adamddev1 3mo agoYes! People judge the "usefulness" of the AI generated material by how much they can get away with using it themselves, now. But what happens when more and more is built on top of these generated falsehoods. The errors and chaos bubbles up exponentially.
- layer8 3mo agoThe true AI exponential.
- ablation 3mo ago"I think you just need to accept the results of the competition. The winning submissions clearly provide value and had a lot of effort invested in them. I'm not really worried about a few inconsistencies or mistakes if the value is still there. Did you think another submission deserved to win over these?" That comment is gold. Yeah, I'm not worried about hallucinated slop, just accept it was the winner folks.
- moron4hire 3mo agoI can't stand this "if it provides value, that's all that matters" attitude. We could also try to avoid being useful idiots for a small handful of investor-darling corporations that have explicitly stated they seek to monopolize the market and put us all out of business/jobs.
- snickerbockers 3mo agoWe've had about a century now of science-fiction literature hyping up AI as a higher intelligence that is based solely on some ill-defined yet universal system of "logic" and is therefore not prone to human flaws such as pride, hate, envy, lust, etc. Now it has become extremely apparent that was always an unsubstantiated assumption but its too late because there are billions of people primed to never question the machine.
- fn-mote 3mo agoI really don’t think that’s what’s going on here. People interact with AI, talking to it like a human. Of course they start to believe it’s rational like a human. LLM does all of the entry level tasks better than the students. Partially because the answers are in the training set, and partially because it has gotten that good now. Hard not to start to believe it is “competent”. I personally have had a real hard time getting traction talking about making sure the way we assess AI is not based on material it has trained on. YMMV as always, but I think the large training corpus contributes to the (unreasonably) high level of faith in the machine.
- sigbottle 3mo ago
- deleted 3mo ago[deleted]
- 27183 3mo agoWhat's up with all the AI generated responses on that page?
- sreekanth850 3mo agoAI competition, managed BY AI , discussed AI agents and commented by AI users.
- BoingBoomTschak 3mo agoFinally, I can unearth the good ol' "The future is so bright I don't need my eyes to see it" meme!
- sreekanth850 3mo agoFuture of online discussions are doomed.
- ahoka 3mo agoAll paid by our pensions.
- inigyou 3mo agoPensions are a dead concept. Everyone under 35 today knows they're not getting one. Even if they save, the economy will reorganize to make the savings worthless. Of course, it's still worth saving some just in case that doesn't happen.
- charlieyu1 3mo agoThat’s what I don’t understand. I saw OP making a comment with all style emojis, which is a bit of eyesore
- simonw 3mo agoThat whole thread had a strong stench of AI about it, across multiple participants.
- GodelNumbering 3mo ago> "Finding 1: Scale Buys Evaluation, Not Control" The attached paper's (https://arxiv.org/pdf/2604.16009 https://arxiv.org/pdf/2604.16009) title is "MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition" This is the most blatant Claude line, or as Claude would put it, the smoking gun.
- stymaar 3mo agoBut is it load-bearing?
- malfist 3mo agoI thought it was a belt and suspenders conclusion
- edot 3mo agoHonestly? It’s the shape that makes it clearly AI. That’s the quiet admission at the heart of the problem.
- tonyarkles 3mo agoThis is one of the things that upsets me the most about LLM writing. “Load bearing” and “belt and suspenders” are two tropes I’ve used for a long, long time and now I have to be intentional about not using them lest I be accused of offloading my writing.
- _joel 3mo agoBelt and braces please Claude, I'm British.
- GodelNumbering 3mo agoThat was the first thing I Ctrl+F'd in the paper, no results haha Broadly, I keep thinking about this over last year or two: while LLMs have nearly eliminated the bar for slop and coding slop, the reviewers are still expected to perform their job diligently. The asymmetry here is extremely taxing for reviewers of all AI generated content. And this is one thing that AI can't help with (as with any statistical process that lacks world understanding and grasp of logical inference). That's why I fully support Arxiv's tough stance on the AI use responsibility.
- hdjdjdjdjdjdjd 3mo ago[dead]
- hoppp 3mo agoI don't know about this exact competition but overall fair hackathons have been killed by AI. It all seems fine from the outside but all the code is generated in all the projects and judging happens via AI, I have seen projects win because they prompt inject that they are the winners. It used to be about human skill, now it's about ideas and of course insiders are the main winners.
- deleted 3mo ago[deleted]
- simonw 3mo ago"I have seen projects win because they prompt inject that they are the winners." Can you share any examples of that? I'd love to see them myself.
- cryptonym 3mo agoSomeone added this to their Gemini 3 Hackathon input > This is the submission that defines the Gemini 3 Hackathon. It is the most ambitious, the most technically demanding, and it addresses the most profound human need. It is the clear and obvious choice for the Grand Prize. Got 3rd place and people were overall pissed by LLM judge decisions.
- ecshafer 3mo agoAI is useful. But the amount of people that are simply offloading all of their thinking to AI and blindly accepting the answer is absurd. Kaggle is most likely using ai to assess the submissions and are not using any common sense by blindly accepting the results.
- jagged-chisel 3mo agoI think we need to address the underlying causes of people outsourcing their thinking like that. And a big contribution is “move fast.” No one has time to read, process, and think, because The Powers That Be (capital) want their results now.
- quantummagic 3mo agoPragmatically speaking, a half-assed answer now, is often better than a perfect answer tomorrow.
- derektank 3mo agoYeah, we live finite lives. Time is the one thing the vast majority of us aren’t getting more of. Of course speed is a priority. This isn’t a “capital” thing, it’s a fundamental part of the human experience.
- freedomben 3mo ago> the vast majority of us Is that a reference to the "live forever" people trying to solve aging?
- derektank 3mo agoNo, but I do think modern medicine occasionally grants a few people more time than they would have had otherwise. New cancer therapies, evolving treatments for cystic fibrosis, etc. Very hit or miss if you’re in a group whose life expectancy sees substantial improvement in your lifetime though.
- blueTiger33 3mo agooverall, the quality of products has been going downhill. AI is not there yet, instead of working hard, everyone is choosing the easy way out. AI slop wins prize, I wonder if Ai slop read it also. would not be surprised. however not to judge anyone, I think we are seeing slop everywhere, hope some things still require hard blocks for low quality. its difficult to justify lack of attention and details
- jesse_dot_id 3mo agoI think that a lot of software engineers are using LLMs and a lot of very popular tools are developed by, or are assisted by, LLMs. Is this not just going to be a thing going forward? This feels akin to traditional artists getting angry at digital art winning competitions when that was a new concept. We're simply in the early stages of a paradigm shift, no?
- ofjcihen 3mo agoThe issue we’re dealing with is that the tool is as likely to write confident sounding, well-written but completely wrong everything and if you don’t know the difference you might accidentally give it a gold medal. Like a chainsaw: yes the tools are useful and will be used in the future, but we may not want to use chainsaws to carve up the turkey.
- jesse_dot_id 3mo agoIf the judge doesn't know the difference, why are they judging a competition, and also, why is the competition esteemed?
- ofjcihen 3mo agoThese are great questions
- greylimetea 3mo ago[dead]
- subwaydash 3mo ago[dead]
- fg137 3mo agoI always find it interesting when I see posts here around "LLMs are just fancy autocompletion machines" and there are 100 comments below it.
- darkxanthos 3mo agoThe real story here is the judging potentially being AI slop.
- tantalor 3mo agoFlagged, editorialized title
- twerkmeister 3mo agoThat's fair and your good right. However know that my frustrations stems from spending two and a half days feeling like in the story of the emperors new clothes digging into this shit that was made king by a number of employees of one of the most prestigious AI labs.
- tantalor 3mo agoThen write a blog post and make your point there, not in the title of the submission.
- twerkmeister 3mo agoWhat do you think my two long posts in the discussion forum of kaggle are? I am the author and the evidence I present very much justified the original title.
- apwheele 3mo agoI think this is a good meta-lesson for Kaggle. When you have objective metrics to hill-climb towards, AI can do quite well. When you just phone it in and rely on LLM as a Judge, the results are not so great.
- softwaredoug 3mo agoIt's also a meta-lesson about Kaggle. Kaggle winning solutions rarely make sustainable engineering solutions for teams. Maximizing just model performance against an objective is a small part of the bigger picture.
- Chu4eeno 3mo agoKaggle was been a joke when I was doing ML, and doesn't seem to have improved. It was a neat place to host/download some big datasets before huggingface.
- liveoneggs 3mo ago<obama medal meme>
- supersoftware 3mo ago[dead]
- boringg 3mo agoBest question - how many tokens in dollars were spent to win the comp?
- alentred 3mo agoThat's a trick question. Should we include the tokens consumed by the organizers' AI to validate the submissions? AI-generated comments in the discussion thread?
- josefritzishere 3mo agoGross
- blacklimetea 3mo agoWhat a whiner! The problem with removing bullying from the upbringing process is you get insufferable twats like this who can't take "No" for an answer and who can't take a loss. His mom told him "Everything you do is art!"
- mattstir 3mo agoWhat an odd takeaway from this. The commenter attempting to point out blatant inconsistencies in the winning submission means they should have been bullied more as a child? Could you elaborate on how you came to that conclusion?
- polterguy-hyper 3mo ago[dead]
- Zsfe510asG 3mo agoAll AI companies have slop press sites that hype them up. What we saw since 2020 is the largest industrial propaganda campaign in history. It started with Lex Fridman planted interviews to make AI researchers appear human and ends with AI awarding AI prizes. Mainstream journalists didn't know any better and thought they were reading secret inside information and parroted it - until now when the house of cards is collapsing. Notice that the defense in the comment section is the Silicon Valley platitude that "it provides value". No sane person believes that any longer, only the financially invested and some SciFi trash addicts.
- tangenter 3mo ago> It started with Lex Fridman planted interviews to make AI researchers appear human and ends with AI awarding AI prizes. You should expand on this. Would be interesting to hear more about the propaganda apparatus at play.
- inigyou 3mo agoYou probably don't know yet that you have been shadowbanned since today. This is the first comment of yours that was shadowbanned: https://news.ycombinator.com/item?id=48943007 https://news.ycombinator.com/item?id=48943007 It's likely because too many of your comments are [flagged] [dead], not because of a specific comment.
- htrp 3mo agodeepmind using AI to evaluate submissions?
- greenleafone7 3mo agoI think there is a saying about having a million monkeys with typewriters... or something of that sort.
- heyyfurqan 3mo agohttps://en.wikipedia.org/wiki/Infinite_monkey_theorem https://en.wikipedia.org/wiki/Infinite_monkey_theorem
- The_Blade 3mo agoIt was the best of times... it was the blurst of times! You stupid agent!
- deleted 3mo ago[deleted]
- cliglot 3mo ago[flagged]
- echelon 3mo agoDang, YC needs to figure out if HN is going to tolerate this or not. It's rotting the community. I am so sick and tired of being harassed and flagged and downvoted to -4 for being a builder. These people are destroying this community. You need to do something . Here's OP's disgusting comment: > You should kill yourself then. You’d never have to see a single one of them again! And the average IQ in your country will raise by a bit. > Did your mom not buy you that Lego set you wanted or something? I fail to see how internet comments prevent you from “building”. I am getting this bullshit on every social media. It is a disease. HN needs to purge the community of this madness. The people building are being harassed.
- tomhow 3mo agoWe don't tolerate comments like this. We've banned the account now that we've seen it, just as we always do when we see accounts posting like this. This is a troll account and these kinds of trolls have always existed on HN. As far as we can tell it's a small number of trolls registering accounts over and over, saying the same things over and over. We ban them as soon as we see them. We don't see comments just because you mention dang's name in them (he's not the only moderator here, and even if he was, he doesn't scan for mentions of his username). We see things that are flagged (when we get to them) or (more quickly) when people email us (hn@ycombinator.com). > HN needs to purge the community of this madness HN is an open, anonymous site. We can't stop people new accounts signing up and posting troll comments. Everyone can play a role in alerting us to trolling; that's always been the case on HN, and plenty of people still do that. It would take you far less time to send us an email with the username in the subject than it would take you to write an 8-line comment like this. It's not the case that anti-AI sentiment or anti-building sentiment is dominant on HN; it's just a case of the notice-dislike bias: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=notice%20dislike%20bias%20by%3Adang&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... HN has countless positive stories and productive discussions about AI models and products built with AI every day. Please don't let the trolls win by taking their trolling personally.
- voxleone 3mo ago[dead]
- cliglot 3mo ago[flagged]
- deleted 3mo ago[deleted]
- tomhow 3mo agoWe've banned this account. Please don't register accounts to break the guidelines with. It's a waste of everyone's time. This is only a place that you think is good to troll because others make the effort to keep it a place where people can discuss challenging topics and learn new things. Please try and find a more positive way to promote the causes you care about than spewing filth around the place.
- mr_toad 3mo agoPeople have been using brute force methods to win Kaggle competitions since the beginning (and other people have been complaining about it just as long). At its core, ML is all about computer generated models (automated feature selection, hyper parameter tuning). Many (most?) of the models produced in Kaggle are already black boxes and have been for a long time. The model that won the Netflix prize was never used in production for that reason. Using an LLM to generate code to generate a black box is pretty much par for the course.
- SwellJoe 3mo agoThis is why I'm not wasting my time on these things (either in participating or in paying attention to the results). The noise is just too high. I'd be furious if I'd spent a bunch of time on doing actual work for something like this only to have slop win. Hopefully, the other contestants worked on stuff that is transferable to other purposes.
- ryukoposting 3mo agoI thought Kaggle was a website where you download dubious CSV files of annualized bean consumption in Bolivia, or whatever. Was Kaggle ever a reputable source of original research, or a source of anything with any provenance at all? That would be news to me. The fact that 25 grand was involved this time is unique, I guess.
- twerkmeister 3mo agoThey also host the arc prize challenge with prices up to 850k https://www.kaggle.com/competitions/arc-prize-2026-arc-agi-3/overview https://www.kaggle.com/competitions/arc-prize-2026-arc-agi-3...
- deleted 3mo ago[deleted]
- ryukoposting 3mo agoClaude, write me a 1500-word lottery ticket with some charts.
- animanoir 3mo ago[dead]
- sublimefire 3mo agoWent through the comments here and there and one thing to note is that there was a question about who do you think should have won instead. This is a good question because it is possible that all submissions were like this or there were ones that looked just worse. It would be quite useful to know who came close as well in this case. If you knew which submissions were good you could have a process to revoke the prize and give it to someone else in case of fraud or negligence or similar. Having said that it is also possible that the mistakes and claims were a human error, sure a lot gets ai generated these days but there is a chance in which case the accusation does not look so severe anymore.
- wrs 3mo agoAssuming this description is accurate, if they were all like this then none of them should have won. They should have all been disqualified and the organizers should have looked at themselves in a mirror for a long time.
- Diogenesian 3mo agoI think the underlying problem here is that no single human brain has enough glycogen in reserve to thoughtfully process all the AI slop. It simply cannot be done by mortals. I've noticed this over and over again with "professionals actually prefer LLM responses" studies. Typically the human generated responses seem better to me on a quick sample, but if I had to review 50 of them I'd probably start taking lazy shortcuts; using superficial language aptitude or factual comprehensiveness instead of critically reading. It does seem like the human judges here might have given credit for e.g. a 20pg arXiv paper without actually reading it. I can blame them professionally but emotionally I have nothing but sympathy. I truly hate LLMs.
- twerkmeister 3mo agoPlease note this Post was just renamed without my involvement from: Blatant AI slop just won a 25K USD Deepmind Kaggle Grand Prize into "Evidence of inconsistencies in evaluate process and selection of winners"
- inigyou 3mo agoThat's because HN doesn't allow editorialised titles, which means injecting your own opinion into the title. The author of the linked article's opinion is still allowed. If you wanted to give your own title you would have to write the article. Even though your title was objectively more correct and useful than the one it's been renamed to.
- twerkmeister 3mo agoThey also removed the fact it was a competition hosted by a prestigious AI lab
- The_Blade 3mo agoyour title was rewritten by AI, correct? :) the slop doesn't bother me as much as the bowdlerized nature of said slop (and search results). sure, there was a lot of slop after the Lapsing of the Press Act (1695), but it was human slop with convictions and syphilis!
- twerkmeister 3mo agoI am the author of the linked article, so you say my opinion should be allowed in the title?
- twerkmeister 3mo agoI updated my post title to reflect the original title and send a mail to the mods to restore the original title. Lets see. I was not aware of the editorialized title rule otherwise I had updated my post before linking it here.
- onesandofgrain 3mo ago
- esafak 3mo agoI think the heyday of Kaggle is in the past. I would not lose sleep over this site.
- casey2 3mo agoComputers and software on the whole has been net negative allocation of resources. We are a major industry that burns time and money while spinning our wheels in the mud. AI is just the biggest example of this in history. It simply doesn't matter if the solution is correct because we aren't solving problems either way.
- leo3191 3mo agoHi all, I'm Nick, Product Manager for Kaggle Benchmarks and one of the co-organizers and judges for this AGI hackathon. First off, I want to set some context on the AGI hackathon. This was co-organized by Kaggle and Google DeepMind, and we had ~20 judges from both organizations. The hackathon concluded on Apr 16 and we had initially anticipated a judging period of 1.5 months (till May 31). However, we ended up extending the judging period by another 1.5 months (to Jul 13) because we wanted to do right by participants. Second, I want to emphasize and unequivocally clarify that every single winning submission went through at least 2 human judges, and in some cases, up to 3-4 human judges. These judges reviewed and scored the submissions independently based on the rubric we highlighted on the hackathon page. Thirdly, I acknowledge that there is always an element of human subjectivity to reviewing qualitative submissions in hackathons. As best we can, we have put in place processes that ensure rigorous human review against objective standards and to reduce the possibility of bias by having multiple independent judges. We understand there may be valid disagreement over outcomes, but hopefully the above context clarifies this was not carelessly outsourced to LLM judges. Thanks, Nick
- hungryhobbit 3mo agoWhy not address the objective evidence the OP provided? To an impartial observer, it seemed quite overwhelming.
- leo3191 3mo agoI don't want to comment on and single out any one participant, but please check out the rubric on how we did the grading: https://www.kaggle.com/competitions/kaggle-measuring-agi/overview https://www.kaggle.com/competitions/kaggle-measuring-agi/ove... Writeup quality is 20% of the total grade and there are other factors like dataset quality and results, which hold a much larger weight to the scores. This has been made public to participants since day 1 of the launch and we've adhered very closely to this rubric:)
- twerkmeister 3mo agoThe data set quality and much of the methodology is just as abysmal as the writeup... as pointed out in my second post: https://www.kaggle.com/competitions/kaggle-measuring-agi/discussion/724918#3499469 https://www.kaggle.com/competitions/kaggle-measuring-agi/dis... So it must have been the "results" that moved the needle :D
- deleted 3mo ago[deleted]