4 ms·

Not sure if this is still true, but it autocompleted some random stripe api private key a few months ago. I have no idea if it was valid. Of course I use enviro

by fswd 4y ago

Not sure if this is still true, but it autocompleted some random stripe api private key a few months ago. I have no idea if it was valid. Of course I use environmental variables.

naikrovek 4y ago
0% chance of validity. GitHub watch for those and notify the provider of the API key so it can be revoked, as well as emailing the owner(s) of the repo which contained the key. if it was in enough places that it got reproduced exactly via neural network, then there is zero chance it escaped their secret-scanners. they even have some GHAS feature that lets you define your own regexes for secrets and will notify you if any are found. none of this makes it safe to store secrets in code, though, these things just help tell you when you've done it.
- jen20 4y ago
  [1] suggests otherwise, at least at launch. If you’re telling me there is NO WAY this could still happen (“zero chance”), that is absolutely a bet I’d place. [1]: https://fossbytes.com/github-copilot-generating-functional-api-keys/ https://fossbytes.com/github-copilot-generating-functional-a...
  - deleted 4y ago
    [deleted]
  - naikrovek 4y ago
    I'm talking about the present, not launch. if I argue that "cars in the US must be equipped with seat belts," would you say "not before 1968!" and believe that we were both talking about the same time period? I guess, as always, I should be absolutely specific about everything, lest someone fill in a gap incorrectly...
    - jen20 4y ago
      I simply don’t believe that sufficient safeguards have been put in place to avoid violating licenses or leaking information. You can choose to, but a bet that there is “zero chance” is pretty foolish. I’ve bookmarked this comment for the inevitable.
      - naikrovek 4y ago
        your idea of inevitable is not really based in reality, I think. but ok. I'll have fun occupying a small portion of your brain for a while.
        jen20 4y ago
        This didn’t take long… https://twitter.com/DocSparse/status/1581461734665367554 https://twitter.com/DocSparse/status/1581461734665367554
phire 4y ago
Probably not valid. With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys. The weights powering the models are only so big, there simply isn't enough memory to waste storing that type of data. Instead, it's learned the general pattern of what an API key looks like and the fact that it's high-entropy random jibberish, with no predictable pattern. And when auto-completing code, it's just going to generate a random string of characters in approximately the correct format.
- avian 4y ago
  > With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys. This can't be true. Copilot used to reproduce code to the inverse square root function from Quake source literally. They blocked it off by blacklisting the function name. Given that, why would learning API keys be impossible? [1] https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287 [2] https://twitter.com/moyix/status/1433261377125326851 https://twitter.com/moyix/status/1433261377125326851
  - sdflhasjd 4y ago
    Because the invsqrt code is well known, and exists copied in multiple places.
  - phire 4y ago
    First, the inverse square root function is not high entropy. At worst it's only medium entropy. Second, it's like the complete opposite of "one-off". It's a block of code that has been copy/pasted into so many code bases, and then there have been hundreds of blog posts, articles and comments written about it. It literally has a Wikipedia article. Most of those places copy/paste the block of code verbatim. It shows up so many times in both the original gpt-3 datasets, and the extra datasets that GitHub did additional training on. That block of code is famous. No wonder why the model though those sequence of tokens were important enough to store and regurgitate verbatim with the minimal amount of prompting. But none of the same applies to an API key found in only one code base.

by fswd 4y ago

Not sure if this is still true, but it autocompleted some random stripe api private key a few months ago. I have no idea if it was valid. Of course I use environmental variables.

naikrovek 4y ago

0% chance of validity. GitHub watch for those and notify the provider of the API key so it can be revoked, as well as emailing the owner(s) of the repo which contained the key. if it was in enough places that it got reproduced exactly via neural network, then there is zero chance it escaped their secret-scanners. they even have some GHAS feature that lets you define your own regexes for secrets and will notify you if any are found. none of this makes it safe to store secrets in code, though, these things just help tell you when you've done it.

jen20 4y ago
[1] suggests otherwise, at least at launch. If you’re telling me there is NO WAY this could still happen (“zero chance”), that is absolutely a bet I’d place. [1]: https://fossbytes.com/github-copilot-generating-functional-api-keys/ https://fossbytes.com/github-copilot-generating-functional-a...
- deleted 4y ago
  [deleted]
- naikrovek 4y ago
  I'm talking about the present, not launch. if I argue that "cars in the US must be equipped with seat belts," would you say "not before 1968!" and believe that we were both talking about the same time period? I guess, as always, I should be absolutely specific about everything, lest someone fill in a gap incorrectly...
  - jen20 4y ago
    I simply don’t believe that sufficient safeguards have been put in place to avoid violating licenses or leaking information. You can choose to, but a bet that there is “zero chance” is pretty foolish. I’ve bookmarked this comment for the inevitable.
    - naikrovek 4y ago
      your idea of inevitable is not really based in reality, I think. but ok. I'll have fun occupying a small portion of your brain for a while.
      - jen20 4y ago
        This didn’t take long… https://twitter.com/DocSparse/status/1581461734665367554 https://twitter.com/DocSparse/status/1581461734665367554

phire 4y ago

Probably not valid. With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys. The weights powering the models are only so big, there simply isn't enough memory to waste storing that type of data. Instead, it's learned the general pattern of what an API key looks like and the fact that it's high-entropy random jibberish, with no predictable pattern. And when auto-completing code, it's just going to generate a random string of characters in approximately the correct format.

avian 4y ago
> With the way machine learning networks work, they don't really learn one-off high-entropy strings like API keys. This can't be true. Copilot used to reproduce code to the inverse square root function from Quake source literally. They blocked it off by blacklisting the function name. Given that, why would learning API keys be impossible? [1] https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287 [2] https://twitter.com/moyix/status/1433261377125326851 https://twitter.com/moyix/status/1433261377125326851
- sdflhasjd 4y ago
  Because the invsqrt code is well known, and exists copied in multiple places.
- phire 4y ago
  First, the inverse square root function is not high entropy. At worst it's only medium entropy. Second, it's like the complete opposite of "one-off". It's a block of code that has been copy/pasted into so many code bases, and then there have been hundreds of blog posts, articles and comments written about it. It literally has a Wikipedia article. Most of those places copy/paste the block of code verbatim. It shows up so many times in both the original gpt-3 datasets, and the extra datasets that GitHub did additional training on. That block of code is famous. No wonder why the model though those sequence of tokens were important enough to store and regurgitate verbatim with the minimal amount of prompting. But none of the same applies to an API key found in only one code base.