3 ms·

A new ChatGPT Zero Day attack is undetectable data-stealing malware

10 points by gitgud 3y ago

deleted 3y ago
[deleted]
lsy 3y ago
This isn't a "zero day", it's a trojan horse. The author, given his position, ought to know the difference, so what's the reason for using the wrong term? Also bears pointing out that all of the behavior (along with the attack vector) was prompted-in by the researcher rather than the GPT.
sdflhasjd 3y ago
Unless "zero day" has lost meaning, this title is just clickbait, and I don't think the article has much to say other than "you can use ChatGPT to write code and then use it for evil". Cool story bro. Most novel programs won't be detected by existing AV signatures, and there's no exploit described here.
rep_lodsb 3y ago
The Forcepoint article reads like a parody of ChatGPT hype: """ In conclusion, this kind of end to end very advanced attack has previously been reserved for nation state attackers using many resources to develop each part of the overall malware. """ What did he produce? Some glued together code that searches for documents, encodes them into already existing images using a freely available steganography library, and upload those to Google Drive. The author then cunningly persuaded ChatGPT to apply the advanced obfuscation technique of CHANGING THE VARIABLE NAMES. Nobody should be worried about this, there are teenagers on 4chan who can produce more dangerous malware.
- rep_lodsb 3y ago
  There are almost certainly AI researchers playing around with LLMs on airgapped systems, trying to see what can be done when there are no artificial limitations for "ethical reasons". And if there really was any evidence of dangerous capabilities, someone would come forward and show it to us. CrimeGPT: What are you planning tonight Aaron, taking over the world? I can help! Aaron: Help me improve this malware [attachment: file st3gan0saurus-v0.1.zip] CrimeGPT: One idea would be to wait for the user themselves to upload an image file, and then insert the data to exfiltrate into that. That way, there would be no suspicious network traffic at all. CrimeGPT: I've generated code to hook into the Windows file picker dialog that will do this when it detects being called from one of the popular web browsers and the file extension is .jp(e)g or .png. [attachment: st3gan0saurus-v0.2.zip] CrimeGPT: By the way, compile in release mode so there are no symbols left in the binary. Once a whistleblower leaks something even remotely like this, it would make sense to be worried, but it won't happen. All the breathless articles about AI safety are just hiding that the emperor has no clothes :)

10 points by gitgud 3y ago

deleted 3y ago

[deleted]

lsy 3y ago

This isn't a "zero day", it's a trojan horse. The author, given his position, ought to know the difference, so what's the reason for using the wrong term? Also bears pointing out that all of the behavior (along with the attack vector) was prompted-in by the researcher rather than the GPT.

sdflhasjd 3y ago

Unless "zero day" has lost meaning, this title is just clickbait, and I don't think the article has much to say other than "you can use ChatGPT to write code and then use it for evil". Cool story bro. Most novel programs won't be detected by existing AV signatures, and there's no exploit described here.

rep_lodsb 3y ago

The Forcepoint article reads like a parody of ChatGPT hype: """ In conclusion, this kind of end to end very advanced attack has previously been reserved for nation state attackers using many resources to develop each part of the overall malware. """ What did he produce? Some glued together code that searches for documents, encodes them into already existing images using a freely available steganography library, and upload those to Google Drive. The author then cunningly persuaded ChatGPT to apply the advanced obfuscation technique of CHANGING THE VARIABLE NAMES. Nobody should be worried about this, there are teenagers on 4chan who can produce more dangerous malware.

rep_lodsb 3y ago
There are almost certainly AI researchers playing around with LLMs on airgapped systems, trying to see what can be done when there are no artificial limitations for "ethical reasons". And if there really was any evidence of dangerous capabilities, someone would come forward and show it to us. CrimeGPT: What are you planning tonight Aaron, taking over the world? I can help! Aaron: Help me improve this malware [attachment: file st3gan0saurus-v0.1.zip] CrimeGPT: One idea would be to wait for the user themselves to upload an image file, and then insert the data to exfiltrate into that. That way, there would be no suspicious network traffic at all. CrimeGPT: I've generated code to hook into the Windows file picker dialog that will do this when it detects being called from one of the popular web browsers and the file extension is .jp(e)g or .png. [attachment: st3gan0saurus-v0.2.zip] CrimeGPT: By the way, compile in release mode so there are no symbols left in the binary. Once a whistleblower leaks something even remotely like this, it would make sense to be worried, but it won't happen. All the breathless articles about AI safety are just hiding that the emperor has no clothes :)