4 ms·

Curiously it focuses on overly descriptive phrasing, and factually incorrect statements, as signs of AI. I don't think this is accurate. AI has a flavour or to

Curiously it focuses on overly descriptive phrasing, and factually incorrect statements, as signs of AI. I don't think this is accurate. AI has a flavour or tone we all know, but it could have generated factually plausible statements (that you could not diagnose in this test) or plausible text. I could not tell the real from fake music at all. I support (and pay for) Kagi, but wasn't overly impressed here. At worst I think it might give people too much confidence. Wikipedia has a great guideline on spotting AI text and I think the game here should integrate and reflect its contents: https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing

A_D_E_P_T 11mo ago
Right. Its examples fall into categories like: - AI slop is trivially factually wrong, and frequently overconfident. - AI slop is verbose. But, as you note, IRL this is not usually the case. It might have been true in the GPT-3.5 or early GPT-4 days, but things have moved on. GPT-5.1 Pro can be laconic and is rarely factually wrong. The best way to identify AI slop text is by their use of special and nonstandard characters. A human would usually write "Gd2O3" for gadolinium oxide, whereas an AI would default to "Gd₂O₃". Chat-GPT also loves to use the non-breaking hyphen (U+2011), whereas all humans typically use the standard hyphen-minus character (U+002D). There's more along these lines. The issue is that the bots are too scrupulously correct in the characters they use. As for music, it can be very tough to distinguish. Interestingly, there are some genres of music that are entirely beyond the ability of AI to replicate.
- CamperBob2 11mo ago
  Interestingly, there are some genres of music that are entirely beyond the ability of AI to replicate. Sounds interesting, what are some of those genres?
  - A_D_E_P_T 11mo ago
    You'll find this surprising, but Suno is completely and utterly incapable of generating ambient music like this: https://www.youtube.com/watch?v=SsA-jxQsx8I https://www.youtube.com/watch?v=SsA-jxQsx8I It simply doesn't get it. This sort of thing probably wasn't in its training data. The really interesting thing is that when I upload something like that track, and tell it to compose something similar, it usually gives me an error and refunds my credits. Also, and this is far more mainstream, both Suno and ElevenLabs are totally incapable of generating anything like, e.g., Darkthrone's "Transylvanian Hunger." Music that is intentionally unpolished is anathema to them. I could go on. There are lots. I think that they understand melody and harmony, but they don't understand atmosphere, just in general...
- vanderZwan 11mo ago
  > But, as you note, IRL this is not usually the case. Except for the huge the amounts of already generated slop that is combined with SEO to pop up in search results
  - visarga 11mo ago
    Oh, if you finetune GPT-4 on an author it assumes the style so well that people prefer it to human experts doing the same job > "Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers"
- Der_Einzige 11mo ago
  I'm like 99.99999% sure that the usage of non standard characters, em dash, fancy quotes, "it's not X, it's Y" etc was clearly done on purpose from the very beginning and pushed for by various parties who have a strong vested interest in monitoring who is using AI and how it's being used. I kneel Hideo Kojima. You saw this all coming: https://youtu.be/PnnP4sA80D8 https://youtu.be/PnnP4sA80D8
- daotoad 11mo ago
  > whereas all humans typically use the standard hyphen-minus character (U+002D). I made it a point to learn to type the em dash—only to have it stolen by the bots; it's forced me to become reacquainted with my long lost friend, the semicolon.
  - A_D_E_P_T 11mo ago
    Hah, well, there's that. But I was referring to the special hyphen that the AIs frequently use today, and which is a hallmark of AI generated text, as it's not on regular keyboards and difficult to access: https://en.wikipedia.org/wiki/Wikipedia:Non-breaking_hyphen https://en.wikipedia.org/wiki/Wikipedia:Non-breaking_hyphen They're also fond of this apostrophe: ’ Whereas almost every human uses: '
    - pajko 11mo ago
      Word automatically corrects apostrophes and converts hyphens to em-dashes
benrutter 11mo ago
> I could not tell the real from fake music at all. Not sure if it's in my head (haven't done a blind test or anything) but all AI music I've heard has painfully bad drum acoustics (very clicky). Seems like the most tell tale marker, although would love to make a "spot the AI song" game to prove myself right/wrong.
- 1970-01-01 11mo ago
  Yes, it's an uncanny valley when hearing sound details along with the digital artifact noise that gives it away. AI generated audio is lagging quite a bit behind AI video and images.
socalgal2 11mo ago
I found the music trivially distiguishable or at least I thought it was. Maybe I just got lucky. The AI songs had lyrics that, to me, seemed like something that wouldn't be in most songs. I've heard far better AI generated songs that I'd have a hard time distguishing if I wasn't told.
alfon 11mo ago
But then it wouldn't classify as slop?
ignoramous 11mo ago
> I support (and pay for) Kagi, but wasn't overly impressed here This website strikes me as merely a marketing gimmick.
- ants_everywhere 11mo ago
  Most likely they see AI as a competitor to search and are trying to survive by pandering to the anti-AI movement
beepbooptheory 11mo ago
> I could not tell the real from fake music at all. Perhaps this is just a sign for you to listen to more (human) music is all!
tvshtr 11mo ago
So what happens if AI is provided with that Wikipedia entry as a guideline of what not to do?

A_D_E_P_T 11mo ago

Right. Its examples fall into categories like: - AI slop is trivially factually wrong, and frequently overconfident. - AI slop is verbose. But, as you note, IRL this is not usually the case. It might have been true in the GPT-3.5 or early GPT-4 days, but things have moved on. GPT-5.1 Pro can be laconic and is rarely factually wrong. The best way to identify AI slop text is by their use of special and nonstandard characters. A human would usually write "Gd2O3" for gadolinium oxide, whereas an AI would default to "Gd₂O₃". Chat-GPT also loves to use the non-breaking hyphen (U+2011), whereas all humans typically use the standard hyphen-minus character (U+002D). There's more along these lines. The issue is that the bots are too scrupulously correct in the characters they use. As for music, it can be very tough to distinguish. Interestingly, there are some genres of music that are entirely beyond the ability of AI to replicate.

CamperBob2 11mo ago
Interestingly, there are some genres of music that are entirely beyond the ability of AI to replicate. Sounds interesting, what are some of those genres?
- A_D_E_P_T 11mo ago
  You'll find this surprising, but Suno is completely and utterly incapable of generating ambient music like this: https://www.youtube.com/watch?v=SsA-jxQsx8I https://www.youtube.com/watch?v=SsA-jxQsx8I It simply doesn't get it. This sort of thing probably wasn't in its training data. The really interesting thing is that when I upload something like that track, and tell it to compose something similar, it usually gives me an error and refunds my credits. Also, and this is far more mainstream, both Suno and ElevenLabs are totally incapable of generating anything like, e.g., Darkthrone's "Transylvanian Hunger." Music that is intentionally unpolished is anathema to them. I could go on. There are lots. I think that they understand melody and harmony, but they don't understand atmosphere, just in general...
vanderZwan 11mo ago
> But, as you note, IRL this is not usually the case. Except for the huge the amounts of already generated slop that is combined with SEO to pop up in search results
- visarga 11mo ago
  Oh, if you finetune GPT-4 on an author it assumes the style so well that people prefer it to human experts doing the same job > "Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers"
Der_Einzige 11mo ago
I'm like 99.99999% sure that the usage of non standard characters, em dash, fancy quotes, "it's not X, it's Y" etc was clearly done on purpose from the very beginning and pushed for by various parties who have a strong vested interest in monitoring who is using AI and how it's being used. I kneel Hideo Kojima. You saw this all coming: https://youtu.be/PnnP4sA80D8 https://youtu.be/PnnP4sA80D8
daotoad 11mo ago
> whereas all humans typically use the standard hyphen-minus character (U+002D). I made it a point to learn to type the em dash—only to have it stolen by the bots; it's forced me to become reacquainted with my long lost friend, the semicolon.
- A_D_E_P_T 11mo ago
  Hah, well, there's that. But I was referring to the special hyphen that the AIs frequently use today, and which is a hallmark of AI generated text, as it's not on regular keyboards and difficult to access: https://en.wikipedia.org/wiki/Wikipedia:Non-breaking_hyphen https://en.wikipedia.org/wiki/Wikipedia:Non-breaking_hyphen They're also fond of this apostrophe: ’ Whereas almost every human uses: '
  - pajko 11mo ago
    Word automatically corrects apostrophes and converts hyphens to em-dashes

benrutter 11mo ago

> I could not tell the real from fake music at all. Not sure if it's in my head (haven't done a blind test or anything) but all AI music I've heard has painfully bad drum acoustics (very clicky). Seems like the most tell tale marker, although would love to make a "spot the AI song" game to prove myself right/wrong.

1970-01-01 11mo ago
Yes, it's an uncanny valley when hearing sound details along with the digital artifact noise that gives it away. AI generated audio is lagging quite a bit behind AI video and images.

socalgal2 11mo ago

I found the music trivially distiguishable or at least I thought it was. Maybe I just got lucky. The AI songs had lyrics that, to me, seemed like something that wouldn't be in most songs. I've heard far better AI generated songs that I'd have a hard time distguishing if I wasn't told.

alfon 11mo ago

But then it wouldn't classify as slop?

ignoramous 11mo ago

> I support (and pay for) Kagi, but wasn't overly impressed here This website strikes me as merely a marketing gimmick.

ants_everywhere 11mo ago
Most likely they see AI as a competitor to search and are trying to survive by pandering to the anti-AI movement

beepbooptheory 11mo ago

> I could not tell the real from fake music at all. Perhaps this is just a sign for you to listen to more (human) music is all!

tvshtr 11mo ago

So what happens if AI is provided with that Wikipedia entry as a guideline of what not to do?