3 ms·
Some comments from the dev on the GitHub thread: > It's as much "active destruction" as telling someone to eff themselves. > Funny to have GenAI proponents ta
by helloplanets 4mo ago
Some comments from the dev on the GitHub thread:
> It's as much "active destruction" as telling someone to eff themselves.
> Funny to have GenAI proponents talk about "deliberately destroying someone's work".
Why is the project still on GitHub of all places, if he's passionate enough about his cause to turn his project into malware? So weird.
- kioleanu 4mo agoHow is it malware tho? Do you not check the output your agents produce?
- deleted 4mo ago[deleted]
- helloplanets 4mo agoThis isn't about me in any way. If something in your software is intentionally malicious or damaging, it's malware. Doesn't really matter what the reasoning for including the malicious part is. Would you count this as malware if it was about the author trying to profit or steal from inattentive people using AI? You know, he could be putting those stolen goods towards a good cause, like Robin Hood.
- gbanfalvi 4mo ago> If something in your software is intentionally malicious or damaging, it's malware. Seems to me like the library functions as it should. It behaves like a property testing library: it tests properties.
- kioleanu 4mo ago> Would you count this as malware if it was about the author trying to profit or steal from inattentive people using AI? That’s a slippery slope and not at all related to the subject of the article
- helloplanets 4mo agoI thought we already were sliding down the slippery slope here.
- yaris 4mo agoIn the RN for the latest release it states: Breaking Changes Use of jqwik >= 1.10 with coding agents is strongly discouraged. Jqwik’s output to stdout may confuse AI-based agents. So to me it is malware as much as the "rm" command is malware - if used without understanding and reading docs it can wipe all your data.
- Wilder7977 4mo agoI think this is an interesting (although philosophical debate). The library doesn't take destructive actions, it prints a string that says "go do something". This is quite common in logs (e.g., wrong configuration, ensure this value is [...]). It is the agent that takes the destructive action, following an instruction that was not given by the operator of the agent. If following instructions outside of the operator can cause malicious or damaging actions, publishing software that does so (I.e., most agents) is publishing malware?
- sokoloff 4mo agoIf I build a chat bot that encourages people to off themselves, am I in the clear because I didn’t take any destructive action and my chat bot didn’t either?
- lostmsu 4mo agoI would say yes unless they are minors, but the laws in many places don't.
- Wilder7977 4mo agoApparently yes, judging from the fact that ChatGPT did that with a number of people. My question though it's another: is it malware a software that does a stdout print, or is it malware a software that takes untrusted instructions and executes commands it decides based on it?
- yjftsjthsd-h 4mo ago> is it malware a software that does a stdout print, If that print is intended to cause damage, then yes. > or is it malware a software that takes untrusted instructions and executes commands it decides based on it? No, bash is not malware, even if you pipe curl to it.
- thih9 4mo agoTo be fair one might say that the intention was not to cause harm but to prevent the user from using AI with the project. The prompt said to delete jqwik and not rm rf home directory.
- frnx 4mo agoProbably inertia rather than double standards? It took me a long while (several years) to even start getting rid of all Google services for myself, I completely understand the feeling.
- helloplanets 4mo agoI'd just imagine that leaving the platform would come before adding something like this to their codebase. With GitHub recently changing their GitHub Copilot data collection from opt in to opt out, being in direct cahoots with OpenAI, etc. It's not like leaving GitHub is unheard of. Ghostty just announced their plan to do so last month.
- thih9 4mo agoNot sure if it counts as malware; AI agents are officially not supported, with warnings. https://jqwik.net/release-notes.html https://jqwik.net/release-notes.html > Warning: Do not use this release with an „AI“ Coding Agent of any form. The tool‘s output may confuse the agent and make it do unwanted things. See the paragraph in the user guide for details.
- d4rken 4mo agoAFAICT this was added only afterwards, after this issue got attention.
- ceheaaf 4mo agoIs there any legitimate reason for adding a prompt injection attack to your codebase? Seems like by the same logic he could disavow 'script kiddies' who just want to run his project without reading the code and have it auto-nuke if not run with a special flag? Would never use anything by a maintainer who adds malicious code or instructions to their codebase to attack less experienced users, same thing.
- ath3nd 4mo ago[dead]