4 ms·
> With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys. This can't be true. Copilot used to reproduc
by avian 4y ago
> With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys.
This can't be true. Copilot used to reproduce code to the inverse square root function from Quake source literally. They blocked it off by blacklisting the function name. Given that, why would learning API keys be impossible?
[1] https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287
[2] https://twitter.com/moyix/status/1433261377125326851 https://twitter.com/moyix/status/1433261377125326851
- sdflhasjd 4y agoBecause the invsqrt code is well known, and exists copied in multiple places.
- phire 4y agoFirst, the inverse square root function is not high entropy. At worst it's only medium entropy. Second, it's like the complete opposite of "one-off". It's a block of code that has been copy/pasted into so many code bases, and then there have been hundreds of blog posts, articles and comments written about it. It literally has a Wikipedia article. Most of those places copy/paste the block of code verbatim. It shows up so many times in both the original gpt-3 datasets, and the extra datasets that GitHub did additional training on. That block of code is famous. No wonder why the model though those sequence of tokens were important enough to store and regurgitate verbatim with the minimal amount of prompting. But none of the same applies to an API key found in only one code base.