4 ms·
Another bunch of dead give aways in code bases with READMEs is the repetitive: - "No X, No Y, No Z." pattern - "Here is X - it makes Y" The worst and most ob
by rvz 5mo ago
Another bunch of dead give aways in code bases with READMEs is the repetitive:
- "No X, No Y, No Z." pattern
- "Here is X - it makes Y"
The worst and most obvious one is the constant over use of emoji ticks and crosses.
- Retr0id 5mo agoFor calibration purposes, I offer you a pre-LLM README I wrote that includes an em-dash* followed by "No X, No Y, No Z": https://github.com/DavidBuchanan314/stelf-loader https://github.com/DavidBuchanan314/stelf-loader *actually a hyphen but it's functioning as an em dash.
- zamadatix 5mo ago"Hyphen functioning as an em dash" is an expected human thing as it's what's easy to type. It's specifically an actual em dash which got bulldozed, much to the dismay of those who bothered to put the unicode character in.
- edbaskerville 5mo agoIf you read The Mac is Not a Typewriter in 1992—thus burning Option-Shift-hyphen into your typing patterns for life, along with a dogmatic love for serif body fonts—you're the real victim here.
- zamadatix 5mo agoOr those of us that use a full featured editor when writing md! This reminds me of another em dash+AI related topic: I've noticed LLMs have an extreme bias towards spaces around the dash while people can go either way with it.
- rzzzt 5mo agoThere's something similar in Microsoft Word, Ctrl-Alt-Minus on the numpad.
- galleywest200 5mo agoI prefer the double dash "--", but Microsoft products will convert this to a proper em-dash if you press space afterwards, I think...
- Grimblewald 5mo agoDouble should map to endash, tripple for em.
- brycewray 5mo agoWas always one of the first things I would change in Word's autocorrect settings whenever I started with a company-provided installation.
- Retr0id 5mo agoA lot of the LLM bots on HN (and elsewhere) will find-and-replace their em dashes with hypens in an attempt to evade detection.
- zamadatix 5mo agoPrecisely, anything to remove AI smells in favor of natural looking text.
- Retr0id 5mo agoMy point is I don't consider em dash vs hyphen to be a strong signal either way, humans and bots alike use both interchangeably.
- zamadatix 5mo agoA signal is not the same thing as a guarantee. Both of your points so far, i.e. your provided text & that bots often bother to replace em dashes to avoid detection, actually support that it is a signal though.
- Retr0id 5mo agoThe stronger signal is the grammatical structure, not the specific glyph used.
- zamadatix 5mo agoThe stronger yet signal is both combined! This glyph, that emoji, a given sentence structure, that formatting, a certain phrase. The more you notice -> the stronger the signal, the more you miss/discard -> the weaker the signal.
- edbaskerville 5mo agoand we will now hold you responsible!
- Grimblewald 5mo agoAlternatively, no one sounds like an llm, an llm sounds like someone, typically those close to the median of the training corpus. If AI were genuinly capable of novelty, it would be a big deal, tech bros having enough work ethic to design new detectable prose for an llm is a mssive reach and has no real evidence supporting it, else why do tech bros only tackle the easier issues? Things we have massive well labelled corpi for? Why is it never dishwashing and folding laundry? I put to you, if you see a trope in AI writing it's because that trope appeared in the training corpus. Therefore, sure, being predjudice against it lets you catch some AI, but you'll also flag human outout. I think that may not be worth it in the end.
- mrob 5mo agoShow me a single substantial (5000+ words) piece of writing from before the release of GPT-3 that triggers Pangram with high confidence.
- Grimblewald 5mo agoBurden of proof that ai tools aren't dogshit isn't really on me, so pipe down and use a more reasonable register you pissant. This site, even this thread is filled with evidence. You're in no position to demand anything, especially not something apparently demanded in bad faith. A request for info I'd be happy to meet, but not this. There are many cases out there of people proving that it does happen. I have personal text not published that do trigger all detectors between 70-100% AI depending on the tool. I wont be sharing these, as "providers" of these "tools" would simply add it to training data and continue to merrily overfit. Bottom line, transformers regress to the mean like many other models, if you as a person produce output aligned with mean of corpus, you'll trigger detection. More importantly, evading detectors is trivial. Find a corpus of text from an author, get an llm to write a note on style, parlance, habits in writing etc. and then use that voice file to drive outputs from an llm. If the source text didnt register as AI, the new ai output also reliably avoids detection. So my problem is, detection doesnt work, false positives are fact, so these tools at best offer harm. A bigger problem¸ undermining your request for proof beyond that which many before me, and including me, burned in an effort to make folks see reason, is that even if i handed you proof on a silver platter you wouldnt understand it. If you could, you'd already understand, because the problem and the math are quite simple. Every example text i've offered in the past now registers 100% human, but many I kept to myself continue to show the same problem, and that never changed. So why would I waste my few remaining tools for sanity checking, when all i can expect from that endevour is losing a tool and shifting nothing in the conversation? No, best I keep that to myself for now.
- vonunov 5mo ago/* This function doesn't return an int. It doesn't return a float. It doesn't return a char. It doesn't ret-- */
- badc0ffee 5mo agoI've seen AI-generated API documentation do this: "Good for X and Y. Do NOT use for Z - use foo instead." A human writer would add that sparingly, for cases where there might be some ambiguity. But the bot added that to basically every single API endpoint.