4 ms·
New proposal for LLM-on-LLM wars: Have an automated system that retrieves every front page URL link on HN and feeds it into a "smart" LLM for analysis of whethe
by walrus01 1mo ago
New proposal for LLM-on-LLM wars: Have an automated system that retrieves every front page URL link on HN and feeds it into a "smart" LLM for analysis of whether or not the site html, css, JS and general template layout match that of a known LLM generation engine. Then give it a "probably vibe coded slop" rank number. If you can't beat 'em, let them fight...
- zrail 1mo agoI haven't tried it personally but my understanding from friends is that Pangram is very accurate, at least for now. https://www.pangram.com/ https://www.pangram.com/
- walrus01 1mo agoAs a paid service it probably works great. In the spirit of weaponizing vibecoded slop vs vibecoded slop, I'm fairly sure that I could 'teach' locally hosted Qwen 3.8-Flash-Next to identify vibe coded slop, with a thorough enough system prompt and access to like, playwright and headless chromium. Or since a lot of vibe coded static sites are just plain html, cs, js, a first try attempt via wget to mirror the first 500KB of the site and then analyze it.
- _joel 1mo agoIronic that site looks like it's AI generated too
- koito17 1mo agoIt looks close to the kind of documentation artifacts GPT 5.6 Sol would generate for me. That style is still rare enough that I prefer it over OP's "we installed shadcn + Tailwind and look like every seed-stage startup SaaS from 2024" style.
- runarberg 1mo agoPlease, do not use LLMs to do actual statistics. They suck at doing actual statistics (like they suck at everything else). Use proven traditional statistical methods like supervised learning for this. You can use traditional (not large) language models to tokenize the content and then a supervised learning trained on the most popular models to detect if those models generated the content.