4 ms·

Except "website made by LLM" almost invariably means the software was "written" by an LLM too, and thus is likely full of security vulnerabilities, bugs, and so

Except "website made by LLM" almost invariably means the software was "written" by an LLM too, and thus is likely full of security vulnerabilities, bugs, and so on. It may also mean that github issues are being reviewed by an AI, which opens the door up to the AI to malicious prompting.

mastermage 2mo ago
Dumb question but its because I genuinely do not know better. What is the security risk of what is as far as i understand a mouse driver with config options?
- nobrains 2mo ago
  a mouse is an input device. most security challenges (passwords, etc.) are input via input devices. those are at a risk of being compromised. also clicks on an on-screen keyboard, etc. in fact, if u think about it, input device data (also know as telemetry - ironically, what we are trying to prevent in the first place by the logi app to do) security is a very serious potential attack vector.
- mzronek 2mo ago
  For downloadable software there is the risk of a supply chain attack. The website is altered to provide a download with a malware-infected version of the software. See the recent hack on CPU-Z: https://gist.github.com/N3mes1s/b5b0b96782b9f832819d2db7c6684f84 https://gist.github.com/N3mes1s/b5b0b96782b9f832819d2db7c668...
  - deadbunny 2mo ago
    That's a problem wether AI is involved or not.
    - breezybottom 2mo ago
      Sure, but LLM use is a warning sign of developer laziness.
      - awwaiid 2mo ago
        If they can get impatience and hubris in there then they are doing great! https://thethreevirtues.com/ https://thethreevirtues.com/
soulofmischief 2mo ago
Right you are, Ken. With the amount of new low-effort software we can expect to see over the next decade, taste becomes a hallmark.
colinb 2mo ago
I think I strongly disagree with your first sentence. Strongly. I've worked with enough people with fine coding skills and terrible English to know that the two are not an uncommon combination. Last year, I worked with someone whose first language was something from China - no idea which flavour - and they would often pass their written comms through an LLM. I could tell when they had done so by the onset of sudden grammatical correctness. But I don't think they were doing this with the intention of deceiving, merely of communicating clearly and correctly. Now, I often find Claude's idea of idiomatic prose to be a bit load-bearingly seam-hitting as it lands not this point, but THAT one, but it is probably better than something hacked out by a person who confuses tenses, cases, pronouns and when they can enverb a noun. So I tend towards giving people the benefit of the doubt on this. Lastly, if you think LLM-written software is full of security vulnerabilities and bugs, I have terrible news for you about the state of human-written code. The fact that what we do is often better than nothing at all, is no big recommendation.
- KronisLV 2mo ago
  > I've worked with enough people with fine coding skills and terrible English What about people with fine English and terrible coding skills? What about people with both terrible English and terrible coding skills? > Now, I often find Claude's idea of idiomatic prose to be a bit load-bearingly seam-hitting as it lands not this point, but THAT one, but it is probably better than something hacked out by a person who confuses tenses, cases, pronouns and when they can enverb a noun. So I tend towards giving people the benefit of the doubt on this. Personally, I find AI writing as insufferable as anyone else (though Anthropic's is particularly bad, maybe just due to my familiarity with it), but I wouldn't judge individuals for using it to make communication more readable, rephrase what they mean etc. If anything, any difficulty in reading my prose would support that. > Lastly, if you think LLM-written software is full of security vulnerabilities and bugs, I have terrible news for you about the state of human-written code. The fact that what we do is often better than nothing at all, is no big recommendation. I guess a lot depends on how you use the LLMs (I bet horrible coding skills coincide with horribly lazy and problematic usage of LLMs for development too), but I wonder how humans actually stack up to the slop-machine when it comes to how good or bad the code they produce is on average, since I'm sure that SOTA model code by now tends towards the upper end of that. In other words, if a person wants to limit themselves to avoiding LLM code over fears of it being bad, they would also have to avoid any any all code written by people that essentially produced the problematic training dataset, as well as any programmers with the same capabilities (or lack thereof). You'd basically have to avoid using a lot/most of the software out there if that's your quality standard.
mlrtime 2mo ago
[flagged]
mlfreeman 2mo ago
> Except "website made by LLM" almost invariably means the software was "written" by an LLM too, please provide some kind of statistics to back up this claim.

mastermage 2mo ago

Dumb question but its because I genuinely do not know better. What is the security risk of what is as far as i understand a mouse driver with config options?

nobrains 2mo ago
a mouse is an input device. most security challenges (passwords, etc.) are input via input devices. those are at a risk of being compromised. also clicks on an on-screen keyboard, etc. in fact, if u think about it, input device data (also know as telemetry - ironically, what we are trying to prevent in the first place by the logi app to do) security is a very serious potential attack vector.
mzronek 2mo ago
For downloadable software there is the risk of a supply chain attack. The website is altered to provide a download with a malware-infected version of the software. See the recent hack on CPU-Z: https://gist.github.com/N3mes1s/b5b0b96782b9f832819d2db7c6684f84 https://gist.github.com/N3mes1s/b5b0b96782b9f832819d2db7c668...
- deadbunny 2mo ago
  That's a problem wether AI is involved or not.
  - breezybottom 2mo ago
    Sure, but LLM use is a warning sign of developer laziness.
    - awwaiid 2mo ago
      If they can get impatience and hubris in there then they are doing great! https://thethreevirtues.com/ https://thethreevirtues.com/

soulofmischief 2mo ago

Right you are, Ken. With the amount of new low-effort software we can expect to see over the next decade, taste becomes a hallmark.

colinb 2mo ago

I think I strongly disagree with your first sentence. Strongly. I've worked with enough people with fine coding skills and terrible English to know that the two are not an uncommon combination. Last year, I worked with someone whose first language was something from China - no idea which flavour - and they would often pass their written comms through an LLM. I could tell when they had done so by the onset of sudden grammatical correctness. But I don't think they were doing this with the intention of deceiving, merely of communicating clearly and correctly. Now, I often find Claude's idea of idiomatic prose to be a bit load-bearingly seam-hitting as it lands not this point, but THAT one, but it is probably better than something hacked out by a person who confuses tenses, cases, pronouns and when they can enverb a noun. So I tend towards giving people the benefit of the doubt on this. Lastly, if you think LLM-written software is full of security vulnerabilities and bugs, I have terrible news for you about the state of human-written code. The fact that what we do is often better than nothing at all, is no big recommendation.

KronisLV 2mo ago
> I've worked with enough people with fine coding skills and terrible English What about people with fine English and terrible coding skills? What about people with both terrible English and terrible coding skills? > Now, I often find Claude's idea of idiomatic prose to be a bit load-bearingly seam-hitting as it lands not this point, but THAT one, but it is probably better than something hacked out by a person who confuses tenses, cases, pronouns and when they can enverb a noun. So I tend towards giving people the benefit of the doubt on this. Personally, I find AI writing as insufferable as anyone else (though Anthropic's is particularly bad, maybe just due to my familiarity with it), but I wouldn't judge individuals for using it to make communication more readable, rephrase what they mean etc. If anything, any difficulty in reading my prose would support that. > Lastly, if you think LLM-written software is full of security vulnerabilities and bugs, I have terrible news for you about the state of human-written code. The fact that what we do is often better than nothing at all, is no big recommendation. I guess a lot depends on how you use the LLMs (I bet horrible coding skills coincide with horribly lazy and problematic usage of LLMs for development too), but I wonder how humans actually stack up to the slop-machine when it comes to how good or bad the code they produce is on average, since I'm sure that SOTA model code by now tends towards the upper end of that. In other words, if a person wants to limit themselves to avoiding LLM code over fears of it being bad, they would also have to avoid any any all code written by people that essentially produced the problematic training dataset, as well as any programmers with the same capabilities (or lack thereof). You'd basically have to avoid using a lot/most of the software out there if that's your quality standard.

mlrtime 2mo ago

[flagged]

mlfreeman 2mo ago

> Except "website made by LLM" almost invariably means the software was "written" by an LLM too, please provide some kind of statistics to back up this claim.