3 ms·

Are people really looking though? Are all open source libraries being run through extensive performance profiling to look for known heuristics? Are they being l

by Mtinie 2y ago

Are people really looking though? Are all open source libraries being run through extensive performance profiling to look for known heuristics? Are they being looked at line by line for aberrations? I don’t have confidence that people are looking for evidence of potential exploitation because of reasons like the ones you bring up. So we’re back to we just don’t know.

arp242 2y ago
Most commonly-used projects are watched by a bunch of people, or diffed on updates. These are not in-depth reviews, but should catch most of it. So yes, people are looking, and have been looking for a long time. The reason Jia Tan could do their thing is because 1) the main meat was in a binary test file, 2) the code to use that seemed relatively harmless at a glance, and 3) people were encouraged to use the .tar.gz files instead of git clone. Also you need to actual get maintainer status, which is not as easy as it sounds. I've been thinking of inserting a "// THIS LINE IS MALICIOUS, PLEASE REPORT IF YOU SEE IT" in some of my projects to see how long it would take. I bet it would be pretty fast either after commit or after tagging a release.
- caf 2y ago
  Maybe // this line is an external audit test - a free gift card to the first person to report finding it.
- kjok 2y ago
  > I've been thinking of inserting a "// THIS LINE IS MALICIOUS, PLEASE REPORT IF YOU SEE IT" in some of my projects to see how long it would take. Tools that use LLMs to review code will catch such projects.
caf 2y ago
With hindsight it's not the runtime behaviour of the library that you'd want to test - the weakest point in the chain is where the distributed source .tar.gz can't be regenerated from the project repository.
- josephg 2y ago
  For how many projects is that actually checked? I bet barely any. Its especially difficult because most projects aren't built in a reproducible way. You should be able to uncompress and compare a source tarball. But if you get a binary and the source code used to generated that binary, there's no way to tell that they match.
  - caf 2y ago
    Luckily the source tarball is the more important one to check, because that's the difference between backdooring one distribution and backdooring them all. It's still not trivial because there might well be legitimate processing steps that are used to create the tarball, but it should be doable.
BobbyTables2 2y ago
It’s worse than that, and that wouldn’t be enough. A large class of exploitation methods simply have no performance impact.

by Mtinie 2y ago

arp242 2y ago

Most commonly-used projects are watched by a bunch of people, or diffed on updates. These are not in-depth reviews, but should catch most of it. So yes, people are looking, and have been looking for a long time. The reason Jia Tan could do their thing is because 1) the main meat was in a binary test file, 2) the code to use that seemed relatively harmless at a glance, and 3) people were encouraged to use the .tar.gz files instead of git clone. Also you need to actual get maintainer status, which is not as easy as it sounds. I've been thinking of inserting a "// THIS LINE IS MALICIOUS, PLEASE REPORT IF YOU SEE IT" in some of my projects to see how long it would take. I bet it would be pretty fast either after commit or after tagging a release.

caf 2y ago
Maybe // this line is an external audit test - a free gift card to the first person to report finding it.
kjok 2y ago
> I've been thinking of inserting a "// THIS LINE IS MALICIOUS, PLEASE REPORT IF YOU SEE IT" in some of my projects to see how long it would take. Tools that use LLMs to review code will catch such projects.

caf 2y ago

With hindsight it's not the runtime behaviour of the library that you'd want to test - the weakest point in the chain is where the distributed source .tar.gz can't be regenerated from the project repository.

josephg 2y ago
For how many projects is that actually checked? I bet barely any. Its especially difficult because most projects aren't built in a reproducible way. You should be able to uncompress and compare a source tarball. But if you get a binary and the source code used to generated that binary, there's no way to tell that they match.
- caf 2y ago
  Luckily the source tarball is the more important one to check, because that's the difference between backdooring one distribution and backdooring them all. It's still not trivial because there might well be legitimate processing steps that are used to create the tarball, but it should be doable.

BobbyTables2 2y ago

It’s worse than that, and that wouldn’t be enough. A large class of exploitation methods simply have no performance impact.