5 ms·

It seems like a very sensible answer from copilot since the prompt includes "Q_" which makes it obvious that the programmer is specifically looking for the Quak

It seems like a very sensible answer from copilot since the prompt includes "Q_" which makes it obvious that the programmer is specifically looking for the Quake version of this function. To me it doesn't show that copilot will regurgitate existing code when I don't want it to, just that if I ask it to copy some famous existing code for me it will oblige.

the_mitsuhiko 5y ago
I was able to trigger it without the Q prompt before. It just made the a nicer looking gif that way. I got it to produce more GPL code too, that one is just not entertaining.
cjaybo 5y ago
Apparently you haven't seen many of the demos that people are showing off? Because saying that this only occurs when the author is explicitly asking for copied code is blatantly false.
- Thomashuet 5y ago
  No I haven't. If you think the other demos are more interesting please link to them. I'm just saying that this demo is biased and that we can't draw any conclusion from it. Actually the author has just confessed optimizing it for entertainment in a sister comment. That doesn't mean that the claim is false but it doesn't show that it is true either.
  - the_mitsuhiko 5y ago
    I think you misunderstood my comment. The same code gets generated if you call the function `float fast_rsqrt` or `float fast_isqrt` for instance. I intentionally wanted it to be looking like `Q_rsqrt` so that people pick up on it quicker.
    - heavyset_go 5y ago
      Thanks for making the video in the OP. Do you have more examples like this that I can share with those who don't use Twitter, like a repo or blog post?
advisedwang 5y ago
The claim for AI systems like this is that it has actually learned something and is generating code from scratch. Oftentimes the authors will claim regurgitation is simply not possible, and this example shows that's a lie. Many arguments on the benefits, legality and power of AI systems rely on this claim. To turn around now and say it's OK to regurgitate in the right setting is to move the goalposts.
- caconym_ 5y ago
  > Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowledged the regurgitation problem, and by their account are working on solutions to it (e.g. attribution at suggestion-generation time) that don't involve sweeping it under the rug.
  - rckstr 5y ago
    They did! In the faq which I can't find anymore they said: >GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.
    - caconym_ 5y ago
      This actually seems like an explicit acknowledgement that regurgitation is possible, and not remotely a claim that it is "simply not possible". It stands to reason that cases where people are intentionally trying to produce regurgitation will strongly overlap with the minority of cases where it actually happens. So I think we are probably suffering from some selection bias in discussions on HN and similar forums--that might be unavoidable, and it certainly stimulates some interesting discussion, but we should try to avoid misrepresenting the product as a whole and/or what its creators have said about it.
OskarS 5y ago
You're not wrong, but the very idea that it will regurgitate copyrighted code at all (and especially at this length, word for word), means that it will be totally unacceptable for many places. In fact, it is arguably not acceptable to use anywhere if you care deeply about copyright.

the_mitsuhiko 5y ago

I was able to trigger it without the Q prompt before. It just made the a nicer looking gif that way. I got it to produce more GPL code too, that one is just not entertaining.

cjaybo 5y ago

Apparently you haven't seen many of the demos that people are showing off? Because saying that this only occurs when the author is explicitly asking for copied code is blatantly false.

Thomashuet 5y ago
No I haven't. If you think the other demos are more interesting please link to them. I'm just saying that this demo is biased and that we can't draw any conclusion from it. Actually the author has just confessed optimizing it for entertainment in a sister comment. That doesn't mean that the claim is false but it doesn't show that it is true either.
- the_mitsuhiko 5y ago
  I think you misunderstood my comment. The same code gets generated if you call the function `float fast_rsqrt` or `float fast_isqrt` for instance. I intentionally wanted it to be looking like `Q_rsqrt` so that people pick up on it quicker.
  - heavyset_go 5y ago
    Thanks for making the video in the OP. Do you have more examples like this that I can share with those who don't use Twitter, like a repo or blog post?

advisedwang 5y ago

The claim for AI systems like this is that it has actually learned something and is generating code from scratch. Oftentimes the authors will claim regurgitation is simply not possible, and this example shows that's a lie. Many arguments on the benefits, legality and power of AI systems rely on this claim. To turn around now and say it's OK to regurgitate in the right setting is to move the goalposts.

caconym_ 5y ago
> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowledged the regurgitation problem, and by their account are working on solutions to it (e.g. attribution at suggestion-generation time) that don't involve sweeping it under the rug.
- rckstr 5y ago
  They did! In the faq which I can't find anymore they said: >GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.
  - caconym_ 5y ago
    This actually seems like an explicit acknowledgement that regurgitation is possible, and not remotely a claim that it is "simply not possible". It stands to reason that cases where people are intentionally trying to produce regurgitation will strongly overlap with the minority of cases where it actually happens. So I think we are probably suffering from some selection bias in discussions on HN and similar forums--that might be unavoidable, and it certainly stimulates some interesting discussion, but we should try to avoid misrepresenting the product as a whole and/or what its creators have said about it.

OskarS 5y ago

You're not wrong, but the very idea that it will regurgitate copyrighted code at all (and especially at this length, word for word), means that it will be totally unacceptable for many places. In fact, it is arguably not acceptable to use anywhere if you care deeply about copyright.