4 ms·

> but offer no real value They could offer value, but just rarely, at least with the LLM/model/context they used. > toll it takes to deal with these mind-numb

> but offer no real value They could offer value, but just rarely, at least with the LLM/model/context they used. > toll it takes to deal with these mind-numbing stupidities. Could have a special area for submitting these where AI does the rejection letter and banning.

meindnoch 1y ago
>They could offer value, but just rarely, at least with the LLM/model/context they used. Eating human excrement can also offer value in the form of undigested pieces of corn and other seeds. Are you interested?
- ElFitz 1y ago
  Funnily enough, fecal transplants (Fecal Microbiota Transplants, FMT) are a thing, used to help treat a range of diseases. It’s even being investigated to help treat depression. So…
  - javcasas 1y ago
    I'm sure it does. But would you like one every other week like the llm slop?
    - ElFitz 1y ago
      Honestly, regarding the whole "LLM slop" thing, I don’t care. I get why others do, but I just don’t. I don’t care how that sausage is made. Heck, sometimes gen AI even allows people who otherwise wouldn’t have had the time or skills to come up with funny things. What annoys me is all the spam SEO-gamed websites with low information density drowning the answer I’m actually looking for in pages of empty sentences. When they haven’t just gamed their way to the top of search results without actually containing any answer. And that didn’t need LLMs to exist. Just greed and actors with interests unaligned with mine. Such as Google’s former head of ads, apparently. [0][1] [0]: https://www.wheresyoured.at/the-men-who-killed-google/ https://www.wheresyoured.at/the-men-who-killed-google/ [1]: https://www.wheresyoured.at/requiem-for-raghavan/ https://www.wheresyoured.at/requiem-for-raghavan/
  - Applejinx 1y ago
    Oh, certainly. I know that if I was the test subject, no matter what else happened it wouldn't be the worst thing done to me that day :)
xg15 1y ago
I think looking at one example is useful: https://hackerone.com/reports/2823554 https://hackerone.com/reports/2823554 What they did was: 1) Prompt LLM for a generic description of potential buffer overflows in strcopy() and a generic demonstration code for a buffer overflow. (With no connection to curl or even OpenSSL at all) 2) Present some stack traces and grep results that show usage of strcopy() in curl and OpenSSL. 3) Simply claim that the strcopy() usages from 2) somehow indicate a buffer overflow, with no additional evidence. 4) When called out, just pretend that the demonstrator code from 1) were the evidence, even though it's obvious that it's just a textbook example and doesn't call any code from curl. It's not that they found some potentially dangerous code in curl and didn't go all the way to prove an overflow, which could have at least some value. The entire thing is just bullshit made to look like a vulnerability report. There is nothing behind it at all. Edit: Oh, cherry on top: The demonstrator doesn't even use strcopy() - nor any other kind of buffer overflow. It tries to construct some shellcode in a buffer, then gives up and literally calls execve("/bin/sh")...
- deepdarkforest 1y ago
  > The problem is in strcpy in the src files of curl.. have you seen the exploit code ?????? The worst part is that once they are asked for clarifications by the poor maintainers, they go on offense and become aggressive. Like imagine the nerve of some people, to use LLMs to try to gaslight an actual expert that they made a mistake, and then act annoyed/angry when the expert asks normal questions
  - xg15 1y ago
    Yep. My guess is that the aggression is part of the ruse. Trying to start drama/intimidating the other when your bluff is being called out is the oldest strategy... (You could see a similar pattern in the xz backdoor scheme, where they were deliberately causing distress for the maintainer to lower their guard.) Or maybe the guy here hoped that the reviewers would run the demo - blindly - and then somehow believe it was real? Because it prints some scary messages and then does open a shell. Even if that's the only thing it does...
ndepoel 1y ago
> They could offer value, but just rarely, at least with the LLM/model/context they used. Still a net negative overall, given that you have to spend a lot of effort separating the wheat from the chaff. > Could have a special area for submitting these where AI does the rejection letter and banning. So we'll just have one AI talking to another AI with an indeterminate outcome and nobody learns anything of value. Truly we live in the future!
- javcasas 1y ago
  It can be better. On slop detection, shadowban the offender and have it discuss with two AI "maintainers", and after 30 messages go and reveal the ruse. Then ban.

meindnoch 1y ago

>They could offer value, but just rarely, at least with the LLM/model/context they used. Eating human excrement can also offer value in the form of undigested pieces of corn and other seeds. Are you interested?

ElFitz 1y ago
Funnily enough, fecal transplants (Fecal Microbiota Transplants, FMT) are a thing, used to help treat a range of diseases. It’s even being investigated to help treat depression. So…
- javcasas 1y ago
  I'm sure it does. But would you like one every other week like the llm slop?
  - ElFitz 1y ago
    Honestly, regarding the whole "LLM slop" thing, I don’t care. I get why others do, but I just don’t. I don’t care how that sausage is made. Heck, sometimes gen AI even allows people who otherwise wouldn’t have had the time or skills to come up with funny things. What annoys me is all the spam SEO-gamed websites with low information density drowning the answer I’m actually looking for in pages of empty sentences. When they haven’t just gamed their way to the top of search results without actually containing any answer. And that didn’t need LLMs to exist. Just greed and actors with interests unaligned with mine. Such as Google’s former head of ads, apparently. [0][1] [0]: https://www.wheresyoured.at/the-men-who-killed-google/ https://www.wheresyoured.at/the-men-who-killed-google/ [1]: https://www.wheresyoured.at/requiem-for-raghavan/ https://www.wheresyoured.at/requiem-for-raghavan/
- Applejinx 1y ago
  Oh, certainly. I know that if I was the test subject, no matter what else happened it wouldn't be the worst thing done to me that day :)

xg15 1y ago

I think looking at one example is useful: https://hackerone.com/reports/2823554 https://hackerone.com/reports/2823554 What they did was: 1) Prompt LLM for a generic description of potential buffer overflows in strcopy() and a generic demonstration code for a buffer overflow. (With no connection to curl or even OpenSSL at all) 2) Present some stack traces and grep results that show usage of strcopy() in curl and OpenSSL. 3) Simply claim that the strcopy() usages from 2) somehow indicate a buffer overflow, with no additional evidence. 4) When called out, just pretend that the demonstrator code from 1) were the evidence, even though it's obvious that it's just a textbook example and doesn't call any code from curl. It's not that they found some potentially dangerous code in curl and didn't go all the way to prove an overflow, which could have at least some value. The entire thing is just bullshit made to look like a vulnerability report. There is nothing behind it at all. Edit: Oh, cherry on top: The demonstrator doesn't even use strcopy() - nor any other kind of buffer overflow. It tries to construct some shellcode in a buffer, then gives up and literally calls execve("/bin/sh")...

deepdarkforest 1y ago
> The problem is in strcpy in the src files of curl.. have you seen the exploit code ?????? The worst part is that once they are asked for clarifications by the poor maintainers, they go on offense and become aggressive. Like imagine the nerve of some people, to use LLMs to try to gaslight an actual expert that they made a mistake, and then act annoyed/angry when the expert asks normal questions
- xg15 1y ago
  Yep. My guess is that the aggression is part of the ruse. Trying to start drama/intimidating the other when your bluff is being called out is the oldest strategy... (You could see a similar pattern in the xz backdoor scheme, where they were deliberately causing distress for the maintainer to lower their guard.) Or maybe the guy here hoped that the reviewers would run the demo - blindly - and then somehow believe it was real? Because it prints some scary messages and then does open a shell. Even if that's the only thing it does...

ndepoel 1y ago

> They could offer value, but just rarely, at least with the LLM/model/context they used. Still a net negative overall, given that you have to spend a lot of effort separating the wheat from the chaff. > Could have a special area for submitting these where AI does the rejection letter and banning. So we'll just have one AI talking to another AI with an indeterminate outcome and nobody learns anything of value. Truly we live in the future!

javcasas 1y ago
It can be better. On slop detection, shadowban the offender and have it discuss with two AI "maintainers", and after 30 messages go and reveal the ruse. Then ban.