4 ms·

Interesting side effect of this (to me :-)) is that when Blekko was acquired by IBM I got stuck doing what is called "Blue Washing" where IBM tracks down the li

Interesting side effect of this (to me :-)) is that when Blekko was acquired by IBM I got stuck doing what is called "Blue Washing" where IBM tracks down the licenses, origins, and usages of all the source code the acquired company is bringing to IBM. According to people I worked with when doing this, the entire process and toolset was an outgrowth of the work they did to disprove SCO's claims in the lawsuit. I found it was both invasive and thorough, and generally engineers didn't like it when the answer was "You have to delete that, we can't verify we have rights to it."

politician 4y ago
I wonder if it'll be much easier going forward. Take a suspect piece of code, run it through an LLM to explain it, pass the explanation to another LLM to generate code from the explanation. An automated pipeline for clean room reverse engineering.
- jedberg 4y ago
  As long as you can prove that you had a license to all the code that was used to train the LLM! :)
  - akiselev 4y ago
    Someone would first have to prove you used an LLM, no?
    - dilyevsky 4y ago
      That’s what discovery and depositions are for
  - politician 4y ago
    Hah! It appears that no one training these models is doing that. GPT doesn't generate the correct GPL/MIT/BSD/Apache license information and attribution for the source code it generates from its training set. All of these code-based LLMs are willfully violating copyright already. It's doubtful this goes anywhere in the courts, however.
- hencq 4y ago
  IANAL but it seems that if you rely on looking at the suspect piece of code to see what it's doing (even if done by an LLM), it's not exactly clean room.
  - politician 4y ago
    It's a two-step process. https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design
kkielhofner 4y ago
There’s an entire cottage industry around this. As one example: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html https://www.synopsys.com/software-integrity/security-testing... Frequently used for things like internal audits, compliance, due-diligence during financing, etc. It’s remarkably thorough but not always completely accurate - it flagged an open source project I created that we were using as GPL. I had to correct them, which was entertaining.
- bitcharmer 4y ago
  They are almost always inaccurate. What they always are is expensive
  - kkielhofner 4y ago
    As CTO I generally tracked software and applicable use license(s) across the entire codebase. My internal sheet more or less lined up with their results - with the notable exception being my project I referenced. FWIR it was something like 400 entries and I’d put it at roughly 95% accurate on this anecdotal rough estimate.
    - sgc 4y ago
      That is surprisingly bad. I would expect it to be some sigma level of accuracy for the money they charge. Perhaps I misunderstand the problem.
      - kkielhofner 4y ago
        This was over 10 years ago. Not only is my memory fuzzy I would imagine (hope) it's gotten significantly better since then. Per usual for these kinds of things what you're really paying for is the name and "trust" associated with it. In the grand scheme of things when you're raising tens of millions - billions of dollars the cost of these tools in terms of total services across due diligence is nothing. Additionally, tools like this are selected by the investor and you don't really have much say in the matter. Basically, you use "expensive" $SOLUTION the investor prefers, check the box, and move on to the hundreds/thousands of other due diligence items in the transaction. Where it gets really interesting (to me) are cases with super aggressive investors like Tiger Global and anything in bubble (crypto) where close times are days and almost no due diligence is performed. As one example I doubt anyone used these kinds of tools at FTX...
patwolf 4y ago
I was at IBM around the time of the SCO lawsuit and had to help out with blue washing a few projects. It was also around that time that there had been some high-profile cases of GPL code getting shipped accidentally. We had a tool that basically grepped through all the source for keywords like "Copyright", "License", or "GPL", and then we had to compile a report and get the lawyers to sign off on it. It didn't seem to me to be a very thorough way of proving provenance because it relied on proper attribution.

politician 4y ago

I wonder if it'll be much easier going forward. Take a suspect piece of code, run it through an LLM to explain it, pass the explanation to another LLM to generate code from the explanation. An automated pipeline for clean room reverse engineering.

jedberg 4y ago
As long as you can prove that you had a license to all the code that was used to train the LLM! :)
- akiselev 4y ago
  Someone would first have to prove you used an LLM, no?
  - dilyevsky 4y ago
    That’s what discovery and depositions are for
- politician 4y ago
  Hah! It appears that no one training these models is doing that. GPT doesn't generate the correct GPL/MIT/BSD/Apache license information and attribution for the source code it generates from its training set. All of these code-based LLMs are willfully violating copyright already. It's doubtful this goes anywhere in the courts, however.
hencq 4y ago
IANAL but it seems that if you rely on looking at the suspect piece of code to see what it's doing (even if done by an LLM), it's not exactly clean room.
- politician 4y ago
  It's a two-step process. https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design

kkielhofner 4y ago

There’s an entire cottage industry around this. As one example: https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html https://www.synopsys.com/software-integrity/security-testing... Frequently used for things like internal audits, compliance, due-diligence during financing, etc. It’s remarkably thorough but not always completely accurate - it flagged an open source project I created that we were using as GPL. I had to correct them, which was entertaining.

bitcharmer 4y ago
They are almost always inaccurate. What they always are is expensive
- kkielhofner 4y ago
  As CTO I generally tracked software and applicable use license(s) across the entire codebase. My internal sheet more or less lined up with their results - with the notable exception being my project I referenced. FWIR it was something like 400 entries and I’d put it at roughly 95% accurate on this anecdotal rough estimate.
  - sgc 4y ago
    That is surprisingly bad. I would expect it to be some sigma level of accuracy for the money they charge. Perhaps I misunderstand the problem.
    - kkielhofner 4y ago
      This was over 10 years ago. Not only is my memory fuzzy I would imagine (hope) it's gotten significantly better since then. Per usual for these kinds of things what you're really paying for is the name and "trust" associated with it. In the grand scheme of things when you're raising tens of millions - billions of dollars the cost of these tools in terms of total services across due diligence is nothing. Additionally, tools like this are selected by the investor and you don't really have much say in the matter. Basically, you use "expensive" $SOLUTION the investor prefers, check the box, and move on to the hundreds/thousands of other due diligence items in the transaction. Where it gets really interesting (to me) are cases with super aggressive investors like Tiger Global and anything in bubble (crypto) where close times are days and almost no due diligence is performed. As one example I doubt anyone used these kinds of tools at FTX...

patwolf 4y ago

I was at IBM around the time of the SCO lawsuit and had to help out with blue washing a few projects. It was also around that time that there had been some high-profile cases of GPL code getting shipped accidentally. We had a tool that basically grepped through all the source for keywords like "Copyright", "License", or "GPL", and then we had to compile a report and get the lawyers to sign off on it. It didn't seem to me to be a very thorough way of proving provenance because it relied on proper attribution.