40 ms·

> It’s not emitting “code”, it’s emitting individual tokens ... The “code” is the arrangement of those tokens, but that is determined by the weighting of the wh

> It’s not emitting “code”, it’s emitting individual tokens ... The “code” is the arrangement of those tokens, but that is determined by the weighting of the whole network This theory of operation is not borne out in reality. It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input. It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images. And so, what can be traced back to individual sources? Quite a bit it would seem.

wrs 4y ago
The output may be recognizable by humans as being “the same”, but an ML classifier can look at what is “obviously” a cat and classify it as a turtle. So while your statement is accurate, it doesn’t explain how “traceability” could really work—what is the actual mechanism of “tracing”?
- falcolas 4y ago
  There's a whole field of work around AI Observability - tracing how data moves through a model. It's intended to help identify biases in the input data, and to help answer the question of "why did it make this decision". And if you can trace how data moves through the model, you can identify "that output came from these inputs" and add the associated metadata.
mcintyre1994 4y ago
On your image generation point, something potentially interesting I saw on twitter/Reddit was about artists uploading “no to AI” images to artstation and then that logo almost appearing to vandalise AI generated art. I’m not sure if it’s a real thing or just people on twitter and stuff who don’t like AI art trolling though. Mostly because I’m not sure how up to date some of these models are being kept.
- AJ007 4y ago
  That's a joke/fake
forrestthewoods 4y ago
> It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images. Can you cite sources? I’ve heard this claim repeatedly but have yet to see a good example.
- falcolas 4y ago
  https://youtu.be/kqPKNksl9hk?t=80 https://youtu.be/kqPKNksl9hk?t=80 <- NatGeo cover https://news.ycombinator.com/item?id=33061707 https://news.ycombinator.com/item?id=33061707 <- Watermarks (there's been a few linked here on HN)
  - Filligree 4y ago
    The AI learns to generate watermark-like things, because those exist on a large fraction of its inputs. It doesn't mean the rest of the output existed in the training set. It's perfectly capable of generating a completely novel picture, then slapping a watermark on it.
    - falcolas 4y ago
      My point is that the watermark is deterministic. It is taken whole-cloth from an input, and reproduced as-is on an output. Thus, it is able to be attributed. Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
      - forrestthewoods 4y ago
        > It is taken whole-cloth from an input, and reproduced as-is on an output. I respectfully disagree with this claim. It’s not as-is. It’s remarkably similar. That’s a big difference. > Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination. I also disagree with this. The uncomfortable truth, imho, is that what Stable Diffusion does is FAR closer to what human artists do than we’d like to admit. Human artists are perfectly capable of reproducing copyright infringing images. They just generally choose not to for legal/moral reasons.
sillysaurusx 4y ago
Well yes, if you’re asked to memorize someone’s code, you’ll get surprisingly far too. The fact that models can do this isn’t evidence of anything. It’s a capability (or “tendency” if you overfit too much). I think it’s pretty obvious that if you train a model to reproduce existing work in its entirety, it fails the “sufficiently transformative” test and thus loses legal protection. But there’s nothing stopping you from re-coding existing implementations of GPL’ed code. I used to do it. And your new code has your own chosen license, even if your ideas came from someone else. Are you sure the same logic shouldn’t apply to models?
- visarga 4y ago
  Are you sure you can do that? Because it is trivial to paraphrase code and text.
  - sillysaurusx 4y ago
    Sure. I’ve done it for Bel. (def no (x) (id x nil)) (def atom (x) (no (id (type x) ‘pair))) (def some (x f) (if (no x) nil (f (car x)) x (some (cdr x)))) (def all (x) (if (no x) t (f (car x)) (all (cdr x)) nil))) I don’t even have to pull up bel.bel to know that those are almost perfect replicas. I typed it on my iPad. EDIT: as far as I can tell, the only diff is that all comes before some. https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613& https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613& I could have kept going through most of the implementation.
    - falcolas 4y ago
      I'd wager that most folks would consider the code for some basic list operations to be a little different from outputting a complete specific implementation of a fast square root function with its exact constants, including the comments.
      - sillysaurusx 4y ago
        Why? They’re both basic functions. I’d argue that if pg had left comments and constants, I’d have replicated those too. Perhaps my example isn’t as impressive as the model’s capability, but operationally it’s the same.
rymate1234 4y ago
> It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input. Which makes sense when you consider that the sort of code that is getting reproduced verbatim is usually library functions which developers may copy and paste verbatim comments and all into their project, especially when you prompt the AI with the header of a function that has been copied and pasted often, so the weightings will in that instance be heavily skewed towards reproducing that function
- falcolas 4y ago
  So that should make it easy to attribute, yes?
  - rymate1234 4y ago
    Probably why, like the article says, they're planning to add that > In an attempt to address the issues with open-source licensing, GitHub plans to introduce a new Copilot feature that will “provide a reference for suggestions that resemble public code on GitHub so that you can make a more informed decision about whether and how to use that code,” including “providing attribution where appropriate.” GitHub also has a configurable filter to block suggestions matching public code.
  - visarga 4y ago
    I think harder, as it is spammed around in all directions. It's easier to attribute a unique piece of code that appears in a single repo. But boilerplate functions don't deserve copyright protection as they are not creative. Can I copyright print('hello world!') if I post it in my repo? Do I deserve a citation from now on?
    - falcolas 4y ago
      For better or worse, AI is a combination of machine learning algorithms. And these algorithms are black boxes solely because we don't add observability to them - we aren't looking. But there is a desire to understand why an AI provided the output it did (to increase trust in AI generated output), and so there's a lot of study and work going into adding that observability. Once that's in place, it becomes pretty straightforward to identify which inputs to a model provided what outputs.
safety1st 4y ago
Even if that theory of operation were true, it seems like an implementation detail that would be largely irrelevant in a court of law. Infringement is infringement regardless of how complicated your method for committing it was. If your AI reproduces a copyrighted work verbatim without permission, surely you are committing infringement? If you would be infringing by doing it via copy and paste, I would think a court would be likely to find that you infringed when you used an AI to do it. It does not matter that the algorithm on the backend is fancy...
- falcolas 4y ago
  Personally, I agree with you. Others do not. But irrespective of legality, my point is that attribution is not such a hard problem as people believe, because their mental model of what's going on is much more complicated than reality. I feel they are seeing it as a true AI (mostly non-deterministic and thus hard to observe) and not ML which is deterministic.
Rekksu 4y ago
Do you have a source for this claim? This isn't really how generative models work
- falcolas 4y ago
  https://news.ycombinator.com/item?id=34187038 https://news.ycombinator.com/item?id=34187038 points to the two specific claims WRT stable diffusion. The co-pilot ones are easily found. Quake 3's "fast inverse square root" is the best known instance.
ShamelessC 4y ago
The quote does not make that assertion…

wrs 4y ago

The output may be recognizable by humans as being “the same”, but an ML classifier can look at what is “obviously” a cat and classify it as a turtle. So while your statement is accurate, it doesn’t explain how “traceability” could really work—what is the actual mechanism of “tracing”?

falcolas 4y ago
There's a whole field of work around AI Observability - tracing how data moves through a model. It's intended to help identify biases in the input data, and to help answer the question of "why did it make this decision". And if you can trace how data moves through the model, you can identify "that output came from these inputs" and add the associated metadata.

mcintyre1994 4y ago

On your image generation point, something potentially interesting I saw on twitter/Reddit was about artists uploading “no to AI” images to artstation and then that logo almost appearing to vandalise AI generated art. I’m not sure if it’s a real thing or just people on twitter and stuff who don’t like AI art trolling though. Mostly because I’m not sure how up to date some of these models are being kept.

AJ007 4y ago
That's a joke/fake

forrestthewoods 4y ago

> It's even being seen in image generation, where NatGeo cover images are reproduced in their entirety or where stock photo watermarks are emitted on finished images. Can you cite sources? I’ve heard this claim repeatedly but have yet to see a good example.

falcolas 4y ago
https://youtu.be/kqPKNksl9hk?t=80 https://youtu.be/kqPKNksl9hk?t=80 <- NatGeo cover https://news.ycombinator.com/item?id=33061707 https://news.ycombinator.com/item?id=33061707 <- Watermarks (there's been a few linked here on HN)
- Filligree 4y ago
  The AI learns to generate watermark-like things, because those exist on a large fraction of its inputs. It doesn't mean the rest of the output existed in the training set. It's perfectly capable of generating a completely novel picture, then slapping a watermark on it.
  - falcolas 4y ago
    My point is that the watermark is deterministic. It is taken whole-cloth from an input, and reproduced as-is on an output. Thus, it is able to be attributed. Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination.
    - forrestthewoods 4y ago
      > It is taken whole-cloth from an input, and reproduced as-is on an output. I respectfully disagree with this claim. It’s not as-is. It’s remarkably similar. That’s a big difference. > Stable diffusion is more like a paintshop artist who grabs bits and pieces of other art and melds them together, and less like a painter who creates from their imagination. I also disagree with this. The uncomfortable truth, imho, is that what Stable Diffusion does is FAR closer to what human artists do than we’d like to admit. Human artists are perfectly capable of reproducing copyright infringing images. They just generally choose not to for legal/moral reasons.

sillysaurusx 4y ago

Well yes, if you’re asked to memorize someone’s code, you’ll get surprisingly far too. The fact that models can do this isn’t evidence of anything. It’s a capability (or “tendency” if you overfit too much). I think it’s pretty obvious that if you train a model to reproduce existing work in its entirety, it fails the “sufficiently transformative” test and thus loses legal protection. But there’s nothing stopping you from re-coding existing implementations of GPL’ed code. I used to do it. And your new code has your own chosen license, even if your ideas came from someone else. Are you sure the same logic shouldn’t apply to models?

visarga 4y ago
Are you sure you can do that? Because it is trivial to paraphrase code and text.
- sillysaurusx 4y ago
  Sure. I’ve done it for Bel. (def no (x) (id x nil)) (def atom (x) (no (id (type x) ‘pair))) (def some (x f) (if (no x) nil (f (car x)) x (some (cdr x)))) (def all (x) (if (no x) t (f (car x)) (all (cdr x)) nil))) I don’t even have to pull up bel.bel to know that those are almost perfect replicas. I typed it on my iPad. EDIT: as far as I can tell, the only diff is that all comes before some. https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613& https://sep.yimg.com/ty/cdn/paulgraham/bel.bel?t=1595850613& I could have kept going through most of the implementation.
  - falcolas 4y ago
    I'd wager that most folks would consider the code for some basic list operations to be a little different from outputting a complete specific implementation of a fast square root function with its exact constants, including the comments.
    - sillysaurusx 4y ago
      Why? They’re both basic functions. I’d argue that if pg had left comments and constants, I’d have replicated those too. Perhaps my example isn’t as impressive as the model’s capability, but operationally it’s the same.

rymate1234 4y ago

> It's been clearly displayed that these tools are emitting verbatim copies of existing code (and its comments) in their input. Which makes sense when you consider that the sort of code that is getting reproduced verbatim is usually library functions which developers may copy and paste verbatim comments and all into their project, especially when you prompt the AI with the header of a function that has been copied and pasted often, so the weightings will in that instance be heavily skewed towards reproducing that function

falcolas 4y ago
So that should make it easy to attribute, yes?
- rymate1234 4y ago
  Probably why, like the article says, they're planning to add that > In an attempt to address the issues with open-source licensing, GitHub plans to introduce a new Copilot feature that will “provide a reference for suggestions that resemble public code on GitHub so that you can make a more informed decision about whether and how to use that code,” including “providing attribution where appropriate.” GitHub also has a configurable filter to block suggestions matching public code.
- visarga 4y ago
  I think harder, as it is spammed around in all directions. It's easier to attribute a unique piece of code that appears in a single repo. But boilerplate functions don't deserve copyright protection as they are not creative. Can I copyright print('hello world!') if I post it in my repo? Do I deserve a citation from now on?
  - falcolas 4y ago
    For better or worse, AI is a combination of machine learning algorithms. And these algorithms are black boxes solely because we don't add observability to them - we aren't looking. But there is a desire to understand why an AI provided the output it did (to increase trust in AI generated output), and so there's a lot of study and work going into adding that observability. Once that's in place, it becomes pretty straightforward to identify which inputs to a model provided what outputs.

safety1st 4y ago

Even if that theory of operation were true, it seems like an implementation detail that would be largely irrelevant in a court of law. Infringement is infringement regardless of how complicated your method for committing it was. If your AI reproduces a copyrighted work verbatim without permission, surely you are committing infringement? If you would be infringing by doing it via copy and paste, I would think a court would be likely to find that you infringed when you used an AI to do it. It does not matter that the algorithm on the backend is fancy...

falcolas 4y ago
Personally, I agree with you. Others do not. But irrespective of legality, my point is that attribution is not such a hard problem as people believe, because their mental model of what's going on is much more complicated than reality. I feel they are seeing it as a true AI (mostly non-deterministic and thus hard to observe) and not ML which is deterministic.

Rekksu 4y ago

Do you have a source for this claim? This isn't really how generative models work

falcolas 4y ago
https://news.ycombinator.com/item?id=34187038 https://news.ycombinator.com/item?id=34187038 points to the two specific claims WRT stable diffusion. The co-pilot ones are easily found. Quake 3's "fast inverse square root" is the best known instance.

ShamelessC 4y ago

The quote does not make that assertion…