First, the inverse square root function is not high entropy. At worst it's only medium entropy. Second, it's like the complete opposite of "one-off". It's a bl

First, the inverse square root function is not high entropy. At worst it's only medium entropy. Second, it's like the complete opposite of "one-off". It's a block of code that has been copy/pasted into so many code bases, and then there have been hundreds of blog posts, articles and comments written about it. It literally has a Wikipedia article. Most of those places copy/paste the block of code verbatim. It shows up so many times in both the original gpt-3 datasets, and the extra datasets that GitHub did additional training on. That block of code is famous. No wonder why the model though those sequence of tokens were important enough to store and regurgitate verbatim with the minimal amount of prompting. But none of the same applies to an API key found in only one code base.