5 ms·

I'm trying to put my finger on why this pisses me off so much. Maybe because I'm guessing (I could be wrong) that this is an utterly selfish act, damaging to t

I'm trying to put my finger on why this pisses me off so much. Maybe because I'm guessing (I could be wrong) that this is an utterly selfish act, damaging to the common good, on the part of whoever submitted the CVE. I.e., it's like vandalizing a Habitat for Humanity office.

shadowgovt 3y ago
I can come up with an explanation that doesn't assume ill intent. It's not too terribly hard to run an auto-scanner for a common vulnerability pattern and then hook that scanner up to an LLM to generate English explanations (because, let's call the tech world what it is: well-formed English is likelier to pass the first-pass filter of not being BS than an English-as-a-second-language attempt to explain a problem). Looking at the reporter, they apparently have an undisclosed thank you from Adobe and Toyota. And when they interjected to try and explain an error in their machine mis-stating the name of the reviewer, the text they injected was likely not English-as-first-language. So I can imagine someone basically trying to run an auto-scanner for common vulns to highlight them for various parties to address, because addressing them increases software health globally. Problem is that it's false-positiving on some (admittedly very fragile, in the "only guarded by every human being writing the right code all the time" sense, i.e. the detector's not wrong that if the called function violates contract stuff will break) working code.
- wrs 3y ago
  As I found out when I first put something up for review on HackerOne, there is a whole cottage industry of people just running scanners and blindly reporting whatever comes out. They are doing this to get something on their resume and maybe receive a bounty, not to improve global software health. It’s basically spam, because all the effort is on the part of the receiver to figure out if these reports make any sense. (And of course you can pay H1 to do that for you.)
- flohofwoe 3y ago
  All that person had to do is just read the frigging code and check if the strcpy() can indeed lead to a memory corruption problem (and ideally create a reproducer) instead of wasting the curl maintainer's time with AI-generated bullshit. In this case it's quite obvious that it's an AI generating the bullshit, but we really need a mandatory disclaimer that something was generated by an AI so that a human can immediately break off any interaction instead of wasting time.
arp242 3y ago
Yeah, the selfishness is what pisses me off as well, but also just the sheer pointlessness of it. After the first 2 messages or so it's pretty clear that this is going nowhere fast and that continuing this is pointless. But they go on anyway. Double wank and shit chips.
richbell 3y ago
What's worse, if a person polluting a valuable system with nonsense is knowingly malicious or completely clueless? To me, this is the evolution of script kiddies and beg bounties. https://www.troyhunt.com/beg-bounties/ https://www.troyhunt.com/beg-bounties/
deleted 3y ago
[deleted]
hypeatei 3y ago
It's so lazy and I (probably most here) can spot the AI writing style instantly. They continue to give nonsense answers and waste time of open source maintainers.
- the_af 3y ago
  Something about the excessive politeness and constant apologizing and clarifying gives it away, right? Just wait until people use LLMs with rude, impatient styles and broken English. Maybe an LLM that goes into flamewar tangents, maybe makes unrelated racist remarks, etc. Then we will have truly reached terminal confusion.
  - dexwiz 3y ago
    What makes you think there aren't people out there running models without the corporate safety and alignment layer? For the average person, this is a big ask, but not for a corporate or nation state actor.
    - fnordpiglet 3y ago
      It’s not that big a task as all that. There are a lot of unaligned models available, and user interfaces that aren’t that hard to use. https://github.com/henk717/KoboldAI https://github.com/henk717/KoboldAI
      - dexwiz 3y ago
        Anything that requires reading a github page is a big ask for an "average person." Maybe not for you or me, but we are in a bubble here.
        fnordpiglet 3y ago
        But we also aren’t a megacorp or a nation state, and while as a percentage of the world we aren’t a huge number, as an absolute number there are a lot of people capable of doing this.
armchairhacker 3y ago
I don't like how the person who responded was very friendly and actually looked at the issue. It seemed like he was being taken advantage of. More importantly, I don't want these kinds of reports to discourage friendly, investigative responses. Open source already has a problem, where many newcomers get discouraged by curt responses and their issues not really getting investigated or addressed. Sometimes the responders come off as rude, because they see so many low-effort PRs and issues and feature requests, and have very limited time. But a lot of "low-effort, low-quality" stuff is just submitted by people who are trying to join OSdev, so they deserve friendly responses and actual investigation like this responder did. This stuff just makes maintainers' lives harder and increases the chance that newcomers get unreasonably-harsh responses, discouraging them from making future contributions (which may eventually become useful). Ironically this is even a problem in HackerOne: I've read multiple blog posts where someone submit a CVE to a big company, and the company responded "this isn't a real problem" and just left it unaddressed (sometimes leading to the public disclosure in the blog post).
- pavel_lishin 3y ago
  > I don't like how the person who responded was very friendly and actually looked at the issue. It seemed like he was being taken advantage of. Well, if he was an asshole to everyone and assumed ill intent each time, then he'd get no bug bounties at all. Nor friends.
rich_sasha 3y ago
> I'm trying to put my finger on why this pisses me off so much What I find frustrating when reading LLM output is that the eye glides over it easily, as it all has the right "texture" of text. But after reading a paragraph, you realize there is no content! And you have to squint hard looking for it, and you can't find it. It's exhausting. And here, the person at the sharp end is doing a valuable, unpaid public service, which is adding insult to injury...
- serf 3y ago
  feels like it'll start an arms race with readers employing LLMs to create condensed/sparse summaries of things. I guess the capability of the specific LLMs will dictate whether or not this is a net loss w.r.t. human communication in general.
- godelski 3y ago
  > What I find frustrating when reading LLM output is that the eye glides over it easily, as it all has the right "texture" of text. But after reading a paragraph, you realize there is no content! And you have to squint hard looking for it, and you can't find it. It's exhausting. I'd just like to say that this is a really elegant way of explaining this concept. I'm going to steal it ;) I think this is also why there's a lot of miscommunication about generative models capabilities. I'm focused on image synthesis and then thing I notice is that these look amazing at first glance. Incredible when scrolling through Twitter, Reddit, or whatever. But the longer you look the weirder they tend to get. Not that this always happens, but enough. I just wish we could talk about ML as if there weren't two camps: utter and useless bullshit vs AGI is coming sooner than an Elon promise of self-driving. It's neither of those things and I think we can't talk about what it really is because we want to believe one of the two extremes.
  - pc3k 3y ago
    > But the longer you look the weirder they tend to get. Not that this always happens, but enough. Especially with the HDR-esque glow that shouldn't exist.
deleted 3y ago
[deleted]
rdtsc 3y ago
> I'm trying to put my finger on why this pisses me off so much. There is also the disappointment and embarrassment I feel for the the other members of the programming community. That sounds silly and irrational, I know, but it plays into it somehow. Some languages probably have a convenient term precisely describing that feeling in one word.

shadowgovt 3y ago

I can come up with an explanation that doesn't assume ill intent. It's not too terribly hard to run an auto-scanner for a common vulnerability pattern and then hook that scanner up to an LLM to generate English explanations (because, let's call the tech world what it is: well-formed English is likelier to pass the first-pass filter of not being BS than an English-as-a-second-language attempt to explain a problem). Looking at the reporter, they apparently have an undisclosed thank you from Adobe and Toyota. And when they interjected to try and explain an error in their machine mis-stating the name of the reviewer, the text they injected was likely not English-as-first-language. So I can imagine someone basically trying to run an auto-scanner for common vulns to highlight them for various parties to address, because addressing them increases software health globally. Problem is that it's false-positiving on some (admittedly very fragile, in the "only guarded by every human being writing the right code all the time" sense, i.e. the detector's not wrong that if the called function violates contract stuff will break) working code.

wrs 3y ago
As I found out when I first put something up for review on HackerOne, there is a whole cottage industry of people just running scanners and blindly reporting whatever comes out. They are doing this to get something on their resume and maybe receive a bounty, not to improve global software health. It’s basically spam, because all the effort is on the part of the receiver to figure out if these reports make any sense. (And of course you can pay H1 to do that for you.)
flohofwoe 3y ago
All that person had to do is just read the frigging code and check if the strcpy() can indeed lead to a memory corruption problem (and ideally create a reproducer) instead of wasting the curl maintainer's time with AI-generated bullshit. In this case it's quite obvious that it's an AI generating the bullshit, but we really need a mandatory disclaimer that something was generated by an AI so that a human can immediately break off any interaction instead of wasting time.

arp242 3y ago

Yeah, the selfishness is what pisses me off as well, but also just the sheer pointlessness of it. After the first 2 messages or so it's pretty clear that this is going nowhere fast and that continuing this is pointless. But they go on anyway. Double wank and shit chips.

richbell 3y ago

What's worse, if a person polluting a valuable system with nonsense is knowingly malicious or completely clueless? To me, this is the evolution of script kiddies and beg bounties. https://www.troyhunt.com/beg-bounties/ https://www.troyhunt.com/beg-bounties/

deleted 3y ago

[deleted]

hypeatei 3y ago

It's so lazy and I (probably most here) can spot the AI writing style instantly. They continue to give nonsense answers and waste time of open source maintainers.

the_af 3y ago
Something about the excessive politeness and constant apologizing and clarifying gives it away, right? Just wait until people use LLMs with rude, impatient styles and broken English. Maybe an LLM that goes into flamewar tangents, maybe makes unrelated racist remarks, etc. Then we will have truly reached terminal confusion.
- dexwiz 3y ago
  What makes you think there aren't people out there running models without the corporate safety and alignment layer? For the average person, this is a big ask, but not for a corporate or nation state actor.
  - fnordpiglet 3y ago
    It’s not that big a task as all that. There are a lot of unaligned models available, and user interfaces that aren’t that hard to use. https://github.com/henk717/KoboldAI https://github.com/henk717/KoboldAI
    - dexwiz 3y ago
      Anything that requires reading a github page is a big ask for an "average person." Maybe not for you or me, but we are in a bubble here.
      - fnordpiglet 3y ago
        But we also aren’t a megacorp or a nation state, and while as a percentage of the world we aren’t a huge number, as an absolute number there are a lot of people capable of doing this.

armchairhacker 3y ago

I don't like how the person who responded was very friendly and actually looked at the issue. It seemed like he was being taken advantage of. More importantly, I don't want these kinds of reports to discourage friendly, investigative responses. Open source already has a problem, where many newcomers get discouraged by curt responses and their issues not really getting investigated or addressed. Sometimes the responders come off as rude, because they see so many low-effort PRs and issues and feature requests, and have very limited time. But a lot of "low-effort, low-quality" stuff is just submitted by people who are trying to join OSdev, so they deserve friendly responses and actual investigation like this responder did. This stuff just makes maintainers' lives harder and increases the chance that newcomers get unreasonably-harsh responses, discouraging them from making future contributions (which may eventually become useful). Ironically this is even a problem in HackerOne: I've read multiple blog posts where someone submit a CVE to a big company, and the company responded "this isn't a real problem" and just left it unaddressed (sometimes leading to the public disclosure in the blog post).

pavel_lishin 3y ago
> I don't like how the person who responded was very friendly and actually looked at the issue. It seemed like he was being taken advantage of. Well, if he was an asshole to everyone and assumed ill intent each time, then he'd get no bug bounties at all. Nor friends.

rich_sasha 3y ago

> I'm trying to put my finger on why this pisses me off so much What I find frustrating when reading LLM output is that the eye glides over it easily, as it all has the right "texture" of text. But after reading a paragraph, you realize there is no content! And you have to squint hard looking for it, and you can't find it. It's exhausting. And here, the person at the sharp end is doing a valuable, unpaid public service, which is adding insult to injury...

serf 3y ago
feels like it'll start an arms race with readers employing LLMs to create condensed/sparse summaries of things. I guess the capability of the specific LLMs will dictate whether or not this is a net loss w.r.t. human communication in general.
godelski 3y ago
> What I find frustrating when reading LLM output is that the eye glides over it easily, as it all has the right "texture" of text. But after reading a paragraph, you realize there is no content! And you have to squint hard looking for it, and you can't find it. It's exhausting. I'd just like to say that this is a really elegant way of explaining this concept. I'm going to steal it ;) I think this is also why there's a lot of miscommunication about generative models capabilities. I'm focused on image synthesis and then thing I notice is that these look amazing at first glance. Incredible when scrolling through Twitter, Reddit, or whatever. But the longer you look the weirder they tend to get. Not that this always happens, but enough. I just wish we could talk about ML as if there weren't two camps: utter and useless bullshit vs AGI is coming sooner than an Elon promise of self-driving. It's neither of those things and I think we can't talk about what it really is because we want to believe one of the two extremes.
- pc3k 3y ago
  > But the longer you look the weirder they tend to get. Not that this always happens, but enough. Especially with the HDR-esque glow that shouldn't exist.

deleted 3y ago

[deleted]

rdtsc 3y ago

> I'm trying to put my finger on why this pisses me off so much. There is also the disappointment and embarrassment I feel for the the other members of the programming community. That sounds silly and irrational, I know, but it plays into it somehow. Some languages probably have a convenient term precisely describing that feeling in one word.