5 ms·
Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code. Question: if they had built one using AI teams
by scosman 7mo ago
Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code.
Question: if they had built one using AI teams in both “rooms”, one writing a spec the other implementing, would that be fine? You’d need to verify spec doesn’t include source code, but that’s easy enough.
It seems to mostly follow the IBM-era precedent. However, since the model probably had the original code in its training data, maybe not? Maybe valid for closed source project but not open-source? Interesting question.
- zozbot234 7mo agoThis seems right to me. If you ask a LLM to derive a spec that has no expressive element of the original code (a clean-room human team can carefully verify this), and then ask another instance of the LLM (with fresh context) to write out code from the spec, how is that different from a "clean room" rewrite? The agent that writes the new code only ever sees the spec, and by assumption (the assumption that's made in all clean room rewrites) the spec is purely factual with all copyrightable expression having been distilled out. But the "deriving the spec (and verifying that it's as clean as possible)" is crucial and cannot be skipped!
- oytis 7mo agoIt requires the original project to not be in the training data for the model for it to be a clean room rewrite
- zozbot234 7mo agoThat only matters if expression of the original project really does end up in the rewrite, doesn't it? This can be checked for (by the team with access to the code) and it's also quite unlikely at least. It's not trivial at all to have an LLM replicate their training verbatim: even when feasible (the Harry Potter case, a work that's going to be massively overweighted in training due to its popularity) it takes very specific prompting and hinting.
- oytis 7mo ago> That only matters if expression of the original project really does end up in the rewrite, doesn't it? No, I don't think so. I hate comparing LLMs with humans, but for a human being familiar with the original code might disqualify them from writing a differently-licensed version. Anyway, LLMs are not human, so as many courts confirmed, their output is not copyrightable at all, under any license.
- toyg 7mo agoUh, this is just a curiosity, but do you have a reference for that last argument? If true, it would mean most commercial code being developed today, since it's increasingly AI-generated, would actually be copyright-free. I don't think most Western courts would uphold that position.
- duskdozer 7mo agohttps://news.ycombinator.com/item?id=47232289 https://news.ycombinator.com/item?id=47232289
- pseudalopex 7mo agoThe headline was misleading. The courts avoided to decide what Thaler could have copyrighted because he said he was not the author.
- vkou 7mo ago> That only matters if expression of the original project really does end up in the rewrite, doesn't it? If that were the case, nobody would bother with clean-room rewrites.
- sigseg1v 7mo agoHow would a team verify this for any current model? They would have to observe and control all training data. In practice, any currently available model that is good enough to perform this task likely fails the clean room criteria due to having a copy of the source code of the project it wants to rewrite. At that point it's basically an expensive lossy copy paste.
- zozbot234 7mo agoYou can always verify the output. Unless the problem being solved really is exceedingly specific and non-trivial, it's at least unlikely that the AI will rip off recognizable expression from the original work. The work may be part of the training but so are many millions of completely unrelated works, so any "family resemblance" would have to be there for very specific reasons about what's being implemented.
- nneonneo 7mo agoSomewhat annoyingly, there's been research that suggests that models can pass information to each other via (effectively) steganographic techniques - specific but apparently harmless choices of tokens, wordings, and so on; see https://arxiv.org/abs/1712.02950 https://arxiv.org/abs/1712.02950 and https://alignment.anthropic.com/2025/subliminal-learning/ https://alignment.anthropic.com/2025/subliminal-learning/ for some simple examples. While it feels unlikely that a simple "write this spec from this code" + "write this code from this spec" loop would actually trigger this kind of hiding behaviour, an LLM trained to accurately reproduce code from such a loop definitely would be capable of hiding code details within the spec - and you can't reasonably prove that the frontier LLMs have not been trained to do so.
- actionfromafar 7mo agoYeah I think, the Compaq / IBM precedent can only superficially apply. It would be like having two teams only meet in a room full of documentation - but both teams crammed the source code the day before. (That, the source code you are "reverse engineering" is in the training data.) It doesn't make sense. Also, it's weird that it's okay apparently to use pirated materials to teach an LLM, but maybe not to disseminate what the LLM then tells you.
- swiftcoder 7mo ago> Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code. It doesn't matter how they structure the agents. Since chardet is in the LLM training set, you can't claim any AI implementation thereof is clean room.
- scosman 7mo agoYeah I mention that in the question. Might still be valid for closed source projects (probably is). I think courts would need to weigh in on the open source side. There’s legal precedent is that you can use a derived work to generate a new unique work (the spec derived for the copyrighted code is very much a derived work). There are rulings that LLMs are transformative works, not just copies of training data. LLMs can’t reproduce their entire training set. But this thinking is also ripe for misuse. I could always train or fine-tune a model on the original work so that it can reproduce the original. We quickly get into statistical arguments here. It’s a really interesting question.
- jacquesm 7mo agoI just wrote a long comment about that, but yes, you are on to something here. The key to me is that the LLM itself is a derived work and that by definition it can not produce something original. Which in turn would make profiting off such a derived work created by an automated process from copyrighted works a case of wholesale copyright infringement. If you can get a judge to agree on that I predict the price of RAM will come down again.
- swiftcoder 7mo ago> There’s legal precedent is that you can use a derived work to generate a new unique work (the spec derived for the copyrighted code is very much a derived work) Indeed, but in the clean room scenario, the party who implements the spec has to be a separate entity that has never seen the code. Whether or not the LLM is copyright infringing is a separate question - it definitely has (at least some) familiarity with the code in question, which makes the "clean room" argument an uphill battle
- 7mo ago
- fergie 7mo agoAnswer: probably not, as API-topography is also a part of copyright Edit: this is wrong
- Tiberium 7mo agoDidn't the Google - Oracle case about Java APIs in Android https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_.... directly disprove this?
- looperhacks 7mo agoIn the end, the supreme court case decided that the re-implementation fell under fair use, it did not answer the copyright question.
- Keyframe 7mo agoWasn't Oracle vs Google about all of that?
- scosman 7mo agoThe courts decided that wasn’t true for IBM, Java and many other cases. API typography describes functionality, which isn’t copyrightable (IANAL).
- duskdozer 7mo agoNot if the codebase was included in training the implementer.