8 ms·

Because in the closed source model the frustrated developer that looked into this SSH slowness submits a ticket for the owner of the malicious code to dismiss.

legobmw99 3y ago
This seems completely unrelated to the grandparent comment’s mention of open source LLMs
- deleted 3y ago
  [deleted]
- undersuit 3y ago
  You're right, I read the comment as: > Imagine a future where state actors have hundreds of AI agents fixing bugs, gaining reputation while they slowly introduce backdoors. I really hope open source () succeed. I guess we can only hope verifiable and open source models can counteract the state actors.
cqqxo4zV46cp 3y ago
It’s insane to consider the actual discovery of this to be anything other than a lightning strike. What’s more interesting here is that we can say with near certainty that there are other backdoors like this out there.
- Randalthorro 3y ago
  Time to start looking at similar cases for sure.
matrix_overload 3y ago
Not necessarily. A frustrated developer posts about it, it catches attention of someone who knows how to use Ghidra et al, and it gets dug out quite fast. Except, with closed-source software maintained by a for-profit company, suck cockup would mean a huge reputational hit, with billions of dollars of lost market cap. So, there are very high incentives for companies to vet their devs, have proper code reviews, etc. But with open-source, anyone can be a contributor, everyone is a friend, and nobody is reliably real-world-identifiable. So, carrying out such attacks is easier by orders magnitude.
- yodsanklai 3y ago
  > So, there are very high incentives for companies to vet their devs, have proper code reviews, etc. I'm not sure about that. It takes a few leetcode interviews to get in major tech companies. As for the review process, it's not always thorough (if it looks legit and the tests pass...). However, employees are identifiable and would take huge risk to be caught doing anything fishy.
- Randalthorro 3y ago
  Absolutely not. Getting a job at any critical infrastructure software dev company is easier than contributing to the Linux kernel.
  - OsrsNeedsf2P 3y ago
    Can confirm. I may work at Meta, but I was nearly banned from contributing to an open source project because my commits kept introducing bugs.
- ivlad 3y ago
  We witnessed Juniper generating their VPN keys with Dual EC DRGB, and then the generator constants subverted with Juniper claiming of now knowing how did it happen. I don’t think it affected Juniper firewall business in any significant way.

legobmw99 3y ago

This seems completely unrelated to the grandparent comment’s mention of open source LLMs

deleted 3y ago
[deleted]
undersuit 3y ago
You're right, I read the comment as: > Imagine a future where state actors have hundreds of AI agents fixing bugs, gaining reputation while they slowly introduce backdoors. I really hope open source () succeed. I guess we can only hope verifiable and open source models can counteract the state actors.

cqqxo4zV46cp 3y ago

It’s insane to consider the actual discovery of this to be anything other than a lightning strike. What’s more interesting here is that we can say with near certainty that there are other backdoors like this out there.

Randalthorro 3y ago
Time to start looking at similar cases for sure.

matrix_overload 3y ago

Not necessarily. A frustrated developer posts about it, it catches attention of someone who knows how to use Ghidra et al, and it gets dug out quite fast. Except, with closed-source software maintained by a for-profit company, suck cockup would mean a huge reputational hit, with billions of dollars of lost market cap. So, there are very high incentives for companies to vet their devs, have proper code reviews, etc. But with open-source, anyone can be a contributor, everyone is a friend, and nobody is reliably real-world-identifiable. So, carrying out such attacks is easier by orders magnitude.

yodsanklai 3y ago
> So, there are very high incentives for companies to vet their devs, have proper code reviews, etc. I'm not sure about that. It takes a few leetcode interviews to get in major tech companies. As for the review process, it's not always thorough (if it looks legit and the tests pass...). However, employees are identifiable and would take huge risk to be caught doing anything fishy.
Randalthorro 3y ago
Absolutely not. Getting a job at any critical infrastructure software dev company is easier than contributing to the Linux kernel.
- OsrsNeedsf2P 3y ago
  Can confirm. I may work at Meta, but I was nearly banned from contributing to an open source project because my commits kept introducing bugs.
ivlad 3y ago
We witnessed Juniper generating their VPN keys with Dual EC DRGB, and then the generator constants subverted with Juniper claiming of now knowing how did it happen. I don’t think it affected Juniper firewall business in any significant way.