14 ms·

100 images out of 350,000 that they looked at were memorized. This seems to mostly happen when an image appears frequently (more than 100 times) in the trainin

100 images out of 350,000 that they looked at were memorized. This seems to mostly happen when an image appears frequently (more than 100 times) in the training data, and/or the dataset is small relative to the model.

deleted 4y ago
[deleted]
londons_explore 4y ago
Removing duplicates and very near duplicates ought to be one of the first things done to any training dataset... I wonder why this wasn't done? Too computation heavy?
- Xelynega 4y ago
  If by "computationally heavy" you're talking about the financial overhead of hiring another layer of human filters to filter duplicates out of the dataset.
  - ben_w 4y ago
    That's an unreasonable presumption. Deduplication, to the best of my knowledge, requires every image be compared to every other image. This is necessarily O(n^2) on n images. IIRC the training set is 2.3 billion images, if so that's 0.5 * 5.29e18 comparisons[0], which, if done by humans, would require employing literally all humans for approximately a year even if we compared 12 images per second 24/7. This has to be done computationally, not by humans. [0] half because (a = b) <=> (b = a)
    - raincole 4y ago
      You can roughly classify the images first (we already have very good AI model for this) and use humans as an additional layer. If two images don't get the same top-3 labels from the classifier, the chance that they're duplicates is neglectable.
    - yojo 4y ago
      Not an expert, but I think you can just hash using an image hashing algorithm and bucket by hash. Should be linear in time/space. You can then either have a human check collisions, or just accept a false positive rate and move on. This is a decent write up of someone doing this on a (smaller) dataset. https://towardsdatascience.com/detection-of-duplicate-images-using-image-hash-functions-4d9c53f04a75 https://towardsdatascience.com/detection-of-duplicate-images...
      - bombolo 4y ago
        This doesn't really work if the image is a meme that has been scaled and converted millions of times.
        HelloNurse 4y ago
        Why not? Many meme variants end up in the same bucket, all colliding. They are as duplicates as random rare images. A large bucket can also be inspected by humans or cut up by applying more perceptual hash function and decreasing tolerance, but it would be counterproductive cheating in this case.
        yojo 4y ago
        Perceptual hashes are reasonably robust to small perturbations. Scaling and converting shouldn’t be a problem, though applying filters that change the colors or blur the lines might. If it’s been changed enough that it hashes to a different value, then it might be reasonable to treat it as a different image. At some point a human is also going to say “that’s not the same.” You can always change your hashing algorithm if you find it’s missing too many dupes. Regardless, for the domain we’re talking about (deduping training data), a few false negatives should be acceptable.
        bombolo 4y ago
        If it's more than a few you end up with overfitting though.
    - babel_ 4y ago
      You can lower the bounds significantly. Images do not require comparison to all others, and the presented approach to detecting and deduplicating images in the paper is easily adapted to be near linear (trading for storage or cleverness). Put simply, do you expect google image search to compare your image to every other possible image? No, they're going to embed it to a vector (512d in the paper) and only compare to probable matches; in the paper they start by brute forcing pairwise comparison of the vectors for the dataset, and then use clique finding to go faster when checking their generated images.
  - londons_explore 4y ago
    At the sort of scales of these datasets, there is no way to filter by hand. But there are lots of ways to identify near identical images algorithmically. Typically the process is to download each image, run it through a neural net to make an image embedding vector (a list of a few hundred floats). Save all those in a database. Then, for each image, if it is too close in 'embed space' to another in the database, then it is a duplicate, and should be removed. This algorithm might catch 'duplicates' that it shouldn't, like multiple people taking photos of the eiffel tower from the same public viewpoint. It might miss real duplicates such as an image failing to match with a collage containing the same image. But it's still better than not removing duplicates at all...
- babel_ 4y ago
  The paper itself actually goes to show that this is insufficient (but should still be done, they went from regenerating 1 in 819 to 1 in 1063 images in a small corpus litmus test), and that diffusion is weaker to image extraction attacks than GANs (and seems to be weaker the better it gets). As to why? Lack of care and vigilance, most likely. If they're willing to spend millions of GPU hours training some of these big models, then this wouldn't be a huge cost, and has near linear complexity over the dataset that parallelises trivially. Sadly, vigilance is a finite resource, and that kind of data cleaning is often dismissed in favour of doubling down on the training and assumming it'll be big enough to handle it (however, as has been shown, that's not the case). (Edit: thanks FeepingCreature. The deduplication test used for the numbers above was a separate test in the paper for just this, not indicative of their extraction attack in general, with a small corpus and compared a diffusion model trained on both. So I liken it to a litmus test for the efficacy of deduplication.)
  - FeepingCreature 4y ago
    Note: they went from regenerating 1 in 819 to 1 in 1063 on a relatively small corpus.
  - f38zf5vdt 4y ago
    SD2's dataset was deduplicated prior to training. That's why the paper is about SD1, which was a prototype model completed before Stability even had VC raises.
- KRAKRISMOTT 4y ago
  Could be an overly enthusiastic data augmentation bug
- 6gvONxR4sf7o 4y ago
  The open source community work in ML tends to be terrible at the science and rigor parts of it. You see tons of lazy “we recreated an open version of X” that fails to be remotely as well done as the peer reviewed version, even from the larger community groups.
kilgnad 4y ago
This makes perfect sense and is expected. But you have to know, none of the images are memorized. It's a coincidence. I can explain. Think of the analogy of linear regression. You have X,Y 2D space. X is the input (the English sentence), Y is the output (the image). You use training data, which is a bunch of coordinates in this 2D space (each coordinate representing an <English, image> pair) to generate a best fit line (aka the machine learning model). The properties of that line fit exactly what's going on here. That trend line will not touch most of the coordinates, but it likely will touch a few. When you take an X from the coordinate that touches the line and feed it into the equation of that line (aka model) you will get a Y (an image) that matches the coordinate Y exactly because the coordinate and the line intersect. Makes sense. This is why an image appears to be memorized. But really it's not memorized. The equation of that line is the ONLY thing memorized here. Why does it happen more when the image appears multiple times in the training data? Well this is exactly what happens in least squares analysis. The line is generated from an equation that involves the averages of all the training data (aka coordinates) and if you have many samples of the same coordinate (aka image) you will skew the average and therefore the line towards touching that specific coordinate. There is no complete technical fix for this issue. With enough training data, EVEN when you get rid of duplicates, the line will likely touch or be very close to a coordinate. If you think about it, for a 2D coordinate system you can select a bunch of coordinates that generate a line that the never touches a single coordinate. But this defeats the purpose of machine learning. You're suppose to sample data you have no knowledge about and derive a model from it. If you can already pre-select images that form a model that never touches a coordinate, it means you already have an understanding of the model and can likely just manually tune all the weights by hand to get what you want. As regular humans, we don't have the super-human ability to do this. We can only do it for the simplest 2D example and all the higher dimensional stuff can only be understood by analogy and random sampling. One thing you can do is just add more data to the training set. That will move the trendline and could shift it away from something it touches. But at the same time it could move it towards touching a new coordinate.
- kgwgk 4y ago
  > It's a coincidence. https://en.wikipedia.org/wiki/Pierre_Menard,_Author_of_the_Quixote https://en.wikipedia.org/wiki/Pierre_Menard,_Author_of_the_Q...
dahart 4y ago
A hundred copyright law violations baked into a photocopier would be a lot, right? Shouldn’t we guarantee the number is zero, and not dismiss the problem even if it seems like it should be uncommon? Since the network trainers can’t control what images users produce with the network once it’s public, there’s no ability to predict how often it reproduces a copyrighted image. > This seems to mostly happen Sounds like duplicate images caused a minority ~25% of the cases? (My calculation: 1-819/1063) If so, then the network is memorizing single images, which we know is theoretically possible when images are sparse in the network’s latent space. What needs to happen to prevent this type of memorization?
- FeepingCreature 4y ago
  Duplicate images caused a minority of cases on their reproduction. All duplicates that were found on SD were of repeated images. "In contrast, we failed to identify any memorization when applying the same methodology to Stable Diffusion - even after attempting to extract the 10,000 most-outlier samples." Read, SD does not (seem to) memorize unusual and unique images (even though Imagen does).
  - dahart 4y ago
    That’s all true, you’re right, but what does it mean we should do? Shouldn’t the models actually guarantee they can’t reproduce inputs, given that in the U.S. all images are copyrighted by default? The only images that are legal to copy for training, and legal to distribute in the form of a neural network that can spit them out are the images explicitly licensed, for example, by Creative Commons. The Imagen outlier results validate the accidental memorization of images, even if it’s a small number. And it might be premature to conclude that one paper’s inability to find memorized outliers in SD means that it doesn’t happen. It might be true, but 10k images is less than one ten-thousandth of the training data, and it’s certainly possible that more successful attack methodologies could exist. This represents a single attempt performed under many assumptions and run on a tiny fraction of the inputs, and nothing more. Even if Stable Diffusion doesn’t memorize outliers, or any non-duplicate images, does that matter? SD will be out of date pretty soon and replaced by another network. If they didn’t take care to prevent memorization, if SD’s memorization behavior (or lack thereof) is accidental, then how do we know it won’t happen more often in the next network? Isn’t this a problem that needs to be explicitly addressed, and not just claim it’s uncommon?
wombat_trouble 4y ago
Oh come on. I'm excited about new technologies and I think that image generation can be a net positive for the society, but can we not do that? First, we had people confidently asserting that stuff of this sort absolutely can't happen. Now, we're moving the goalposts to "well, not a legitimate criticism because it doesn't happen often". The point is that basically all Stable Diffusion / DALL-E / MidJourney output is some shade of this; the only new data is that contrary to prior assertions, in some cases, it goes all the way to a verbatim copy. I think there are some defensible stances one can take. One is to reject the idea of intellectual property. Another is to advocate for some specific legal or technical bar that the models would have to pass for it to qualify as "not stealing". Yet another is to argue it's a morally-agnostic technology like VHS or a photocopier, and the burden of using it in a socially acceptable way rests with the user.
- vanderZwan 4y ago
  > and the burden of using it in a socially acceptable way rests with the user. Or, you know, legislation. I'm kind of sick of everything being offloaded as a responsibility of the end-user as an excuse to externalize costs. Plus, in this case it's not even like VHS or a photocopier, it's more like the printing press or the Jacquard loom: those with capital to invest in it benefit the most, at the expense of individuals being exploited.
  - williamcotton 4y ago
    I’d prefer the liability being on the end-user. Tools that are a burden to use, like tools that produce too many infringing works, are not going to sell as well as those that are not a burden to use. This means that if someone makes a tool like this that also alerts the user of likely infringement it would perform better in a corporate, risk-averse marketplace.
- FeepingCreature 4y ago
  > Oh come on. I'm excited about new technologies and I think that image generation can be a net positive for the society, but can we not do that? What, summarize the submission? This is straight quoted from the link. > The point is that basically all Stable Diffusion / DALL-E / MidJourney output is some shade of this Yes, and that point is mistaken, or so generic as to be worthless. The network memorizes art for the same reason humans memorize art: because there's some art pieces we see so often that we can recall them easily. Ask an artist to duplicate Starry Night or Scream from memory, you'll probably get at least a passable imitation. The more capable the artist, the more faithful it will be. We know that SD can be made to plagiarize, given repeated training on a specific image. (This is just to say that a neural network can learn to regurgitate a sample, a capability that was not ever in question.) This is a far cry from the assertions that its art is generally plagiarized.
seanhunter 4y ago
Seems like it would be very simple to prevent this from happening by just adding a perceptual hash check[1] for collisions vs the training set before emitting an image. I'm sure someone would be smart enough to make a "perceptual cuckoo hash" type thing so it would be very fast at the expense of sometimes erring on the side of caution and not emitting an image that actually isn't the same as the training data. I don't know why they didn't do this tbh. [1] https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing
- HelloNurse 4y ago
  Because they don't care. See above comments about legislation.
- crazygringo 4y ago
  Yeah it seems so obvious I can't help but wonder if there's something about it that doesn't work in practice. Like if you set it to a threshold where it's not routinely catching false positives, it doesn't catch the original images either because they're still too subtly different or something. Maybe they really did just not bother though. Heck, maybe they want to test it in court or something.
  - pixl97 4y ago
    I'd say the vast majority of time when working with people programming/testing I hear the statement "Hmm, I didn't think of that".
    - deleted 4y ago
      [deleted]
  - adamsmith143 4y ago
    You can of course just build a classifier to detect training set images and it would do very well even with subtly different images.
  - RC_ITR 4y ago
SkyPuncher 4y ago
Seems to be no different than how people memorize "facts". I'm watching my daughter learn shapes right now. When I see a square, I call it a square and look for a matching "square" piece. It seems when my daughter sees a square, see is still looking at it's properties, labeling individual attributes (pointy thing here), then trying to find the same attributes on the corresponding piece. Eventually, my daughter will see a square enough that the concept will be re-enforced in her mind as a square. I wonder if a similar behavior is happening here. This specific image was presented enough times that it essentially "burned into" the AI. Rather than having to attempt to generate this image from scratch, the AI will simply pull out the canonical reference.
deleted 4y ago
[deleted]

deleted 4y ago

[deleted]

londons_explore 4y ago

Removing duplicates and very near duplicates ought to be one of the first things done to any training dataset... I wonder why this wasn't done? Too computation heavy?

Xelynega 4y ago
If by "computationally heavy" you're talking about the financial overhead of hiring another layer of human filters to filter duplicates out of the dataset.
- ben_w 4y ago
  That's an unreasonable presumption. Deduplication, to the best of my knowledge, requires every image be compared to every other image. This is necessarily O(n^2) on n images. IIRC the training set is 2.3 billion images, if so that's 0.5 * 5.29e18 comparisons[0], which, if done by humans, would require employing literally all humans for approximately a year even if we compared 12 images per second 24/7. This has to be done computationally, not by humans. [0] half because (a = b) <=> (b = a)
  - raincole 4y ago
    You can roughly classify the images first (we already have very good AI model for this) and use humans as an additional layer. If two images don't get the same top-3 labels from the classifier, the chance that they're duplicates is neglectable.
  - yojo 4y ago
    Not an expert, but I think you can just hash using an image hashing algorithm and bucket by hash. Should be linear in time/space. You can then either have a human check collisions, or just accept a false positive rate and move on. This is a decent write up of someone doing this on a (smaller) dataset. https://towardsdatascience.com/detection-of-duplicate-images-using-image-hash-functions-4d9c53f04a75 https://towardsdatascience.com/detection-of-duplicate-images...
    - bombolo 4y ago
      This doesn't really work if the image is a meme that has been scaled and converted millions of times.
      - HelloNurse 4y ago
        Why not? Many meme variants end up in the same bucket, all colliding. They are as duplicates as random rare images. A large bucket can also be inspected by humans or cut up by applying more perceptual hash function and decreasing tolerance, but it would be counterproductive cheating in this case.
      - yojo 4y ago
        Perceptual hashes are reasonably robust to small perturbations. Scaling and converting shouldn’t be a problem, though applying filters that change the colors or blur the lines might. If it’s been changed enough that it hashes to a different value, then it might be reasonable to treat it as a different image. At some point a human is also going to say “that’s not the same.” You can always change your hashing algorithm if you find it’s missing too many dupes. Regardless, for the domain we’re talking about (deduping training data), a few false negatives should be acceptable.
        bombolo 4y ago
        If it's more than a few you end up with overfitting though.
  - babel_ 4y ago
    You can lower the bounds significantly. Images do not require comparison to all others, and the presented approach to detecting and deduplicating images in the paper is easily adapted to be near linear (trading for storage or cleverness). Put simply, do you expect google image search to compare your image to every other possible image? No, they're going to embed it to a vector (512d in the paper) and only compare to probable matches; in the paper they start by brute forcing pairwise comparison of the vectors for the dataset, and then use clique finding to go faster when checking their generated images.
- londons_explore 4y ago
  At the sort of scales of these datasets, there is no way to filter by hand. But there are lots of ways to identify near identical images algorithmically. Typically the process is to download each image, run it through a neural net to make an image embedding vector (a list of a few hundred floats). Save all those in a database. Then, for each image, if it is too close in 'embed space' to another in the database, then it is a duplicate, and should be removed. This algorithm might catch 'duplicates' that it shouldn't, like multiple people taking photos of the eiffel tower from the same public viewpoint. It might miss real duplicates such as an image failing to match with a collage containing the same image. But it's still better than not removing duplicates at all...
babel_ 4y ago
The paper itself actually goes to show that this is insufficient (but should still be done, they went from regenerating 1 in 819 to 1 in 1063 images in a small corpus litmus test), and that diffusion is weaker to image extraction attacks than GANs (and seems to be weaker the better it gets). As to why? Lack of care and vigilance, most likely. If they're willing to spend millions of GPU hours training some of these big models, then this wouldn't be a huge cost, and has near linear complexity over the dataset that parallelises trivially. Sadly, vigilance is a finite resource, and that kind of data cleaning is often dismissed in favour of doubling down on the training and assumming it'll be big enough to handle it (however, as has been shown, that's not the case). (Edit: thanks FeepingCreature. The deduplication test used for the numbers above was a separate test in the paper for just this, not indicative of their extraction attack in general, with a small corpus and compared a diffusion model trained on both. So I liken it to a litmus test for the efficacy of deduplication.)
- FeepingCreature 4y ago
  Note: they went from regenerating 1 in 819 to 1 in 1063 on a relatively small corpus.
- f38zf5vdt 4y ago
  SD2's dataset was deduplicated prior to training. That's why the paper is about SD1, which was a prototype model completed before Stability even had VC raises.
KRAKRISMOTT 4y ago
Could be an overly enthusiastic data augmentation bug
6gvONxR4sf7o 4y ago
The open source community work in ML tends to be terrible at the science and rigor parts of it. You see tons of lazy “we recreated an open version of X” that fails to be remotely as well done as the peer reviewed version, even from the larger community groups.

kilgnad 4y ago

This makes perfect sense and is expected. But you have to know, none of the images are memorized. It's a coincidence. I can explain. Think of the analogy of linear regression. You have X,Y 2D space. X is the input (the English sentence), Y is the output (the image). You use training data, which is a bunch of coordinates in this 2D space (each coordinate representing an <English, image> pair) to generate a best fit line (aka the machine learning model). The properties of that line fit exactly what's going on here. That trend line will not touch most of the coordinates, but it likely will touch a few. When you take an X from the coordinate that touches the line and feed it into the equation of that line (aka model) you will get a Y (an image) that matches the coordinate Y exactly because the coordinate and the line intersect. Makes sense. This is why an image appears to be memorized. But really it's not memorized. The equation of that line is the ONLY thing memorized here. Why does it happen more when the image appears multiple times in the training data? Well this is exactly what happens in least squares analysis. The line is generated from an equation that involves the averages of all the training data (aka coordinates) and if you have many samples of the same coordinate (aka image) you will skew the average and therefore the line towards touching that specific coordinate. There is no complete technical fix for this issue. With enough training data, EVEN when you get rid of duplicates, the line will likely touch or be very close to a coordinate. If you think about it, for a 2D coordinate system you can select a bunch of coordinates that generate a line that the never touches a single coordinate. But this defeats the purpose of machine learning. You're suppose to sample data you have no knowledge about and derive a model from it. If you can already pre-select images that form a model that never touches a coordinate, it means you already have an understanding of the model and can likely just manually tune all the weights by hand to get what you want. As regular humans, we don't have the super-human ability to do this. We can only do it for the simplest 2D example and all the higher dimensional stuff can only be understood by analogy and random sampling. One thing you can do is just add more data to the training set. That will move the trendline and could shift it away from something it touches. But at the same time it could move it towards touching a new coordinate.

kgwgk 4y ago
> It's a coincidence. https://en.wikipedia.org/wiki/Pierre_Menard,_Author_of_the_Quixote https://en.wikipedia.org/wiki/Pierre_Menard,_Author_of_the_Q...

dahart 4y ago

A hundred copyright law violations baked into a photocopier would be a lot, right? Shouldn’t we guarantee the number is zero, and not dismiss the problem even if it seems like it should be uncommon? Since the network trainers can’t control what images users produce with the network once it’s public, there’s no ability to predict how often it reproduces a copyrighted image. > This seems to mostly happen Sounds like duplicate images caused a minority ~25% of the cases? (My calculation: 1-819/1063) If so, then the network is memorizing single images, which we know is theoretically possible when images are sparse in the network’s latent space. What needs to happen to prevent this type of memorization?

FeepingCreature 4y ago
Duplicate images caused a minority of cases on their reproduction. All duplicates that were found on SD were of repeated images. "In contrast, we failed to identify any memorization when applying the same methodology to Stable Diffusion - even after attempting to extract the 10,000 most-outlier samples." Read, SD does not (seem to) memorize unusual and unique images (even though Imagen does).
- dahart 4y ago
  That’s all true, you’re right, but what does it mean we should do? Shouldn’t the models actually guarantee they can’t reproduce inputs, given that in the U.S. all images are copyrighted by default? The only images that are legal to copy for training, and legal to distribute in the form of a neural network that can spit them out are the images explicitly licensed, for example, by Creative Commons. The Imagen outlier results validate the accidental memorization of images, even if it’s a small number. And it might be premature to conclude that one paper’s inability to find memorized outliers in SD means that it doesn’t happen. It might be true, but 10k images is less than one ten-thousandth of the training data, and it’s certainly possible that more successful attack methodologies could exist. This represents a single attempt performed under many assumptions and run on a tiny fraction of the inputs, and nothing more. Even if Stable Diffusion doesn’t memorize outliers, or any non-duplicate images, does that matter? SD will be out of date pretty soon and replaced by another network. If they didn’t take care to prevent memorization, if SD’s memorization behavior (or lack thereof) is accidental, then how do we know it won’t happen more often in the next network? Isn’t this a problem that needs to be explicitly addressed, and not just claim it’s uncommon?

wombat_trouble 4y ago

Oh come on. I'm excited about new technologies and I think that image generation can be a net positive for the society, but can we not do that? First, we had people confidently asserting that stuff of this sort absolutely can't happen. Now, we're moving the goalposts to "well, not a legitimate criticism because it doesn't happen often". The point is that basically all Stable Diffusion / DALL-E / MidJourney output is some shade of this; the only new data is that contrary to prior assertions, in some cases, it goes all the way to a verbatim copy. I think there are some defensible stances one can take. One is to reject the idea of intellectual property. Another is to advocate for some specific legal or technical bar that the models would have to pass for it to qualify as "not stealing". Yet another is to argue it's a morally-agnostic technology like VHS or a photocopier, and the burden of using it in a socially acceptable way rests with the user.

vanderZwan 4y ago
> and the burden of using it in a socially acceptable way rests with the user. Or, you know, legislation. I'm kind of sick of everything being offloaded as a responsibility of the end-user as an excuse to externalize costs. Plus, in this case it's not even like VHS or a photocopier, it's more like the printing press or the Jacquard loom: those with capital to invest in it benefit the most, at the expense of individuals being exploited.
- williamcotton 4y ago
  I’d prefer the liability being on the end-user. Tools that are a burden to use, like tools that produce too many infringing works, are not going to sell as well as those that are not a burden to use. This means that if someone makes a tool like this that also alerts the user of likely infringement it would perform better in a corporate, risk-averse marketplace.
FeepingCreature 4y ago
> Oh come on. I'm excited about new technologies and I think that image generation can be a net positive for the society, but can we not do that? What, summarize the submission? This is straight quoted from the link. > The point is that basically all Stable Diffusion / DALL-E / MidJourney output is some shade of this Yes, and that point is mistaken, or so generic as to be worthless. The network memorizes art for the same reason humans memorize art: because there's some art pieces we see so often that we can recall them easily. Ask an artist to duplicate Starry Night or Scream from memory, you'll probably get at least a passable imitation. The more capable the artist, the more faithful it will be. We know that SD can be made to plagiarize, given repeated training on a specific image. (This is just to say that a neural network can learn to regurgitate a sample, a capability that was not ever in question.) This is a far cry from the assertions that its art is generally plagiarized.

seanhunter 4y ago

Seems like it would be very simple to prevent this from happening by just adding a perceptual hash check[1] for collisions vs the training set before emitting an image. I'm sure someone would be smart enough to make a "perceptual cuckoo hash" type thing so it would be very fast at the expense of sometimes erring on the side of caution and not emitting an image that actually isn't the same as the training data. I don't know why they didn't do this tbh. [1] https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing

HelloNurse 4y ago
Because they don't care. See above comments about legislation.
crazygringo 4y ago
Yeah it seems so obvious I can't help but wonder if there's something about it that doesn't work in practice. Like if you set it to a threshold where it's not routinely catching false positives, it doesn't catch the original images either because they're still too subtly different or something. Maybe they really did just not bother though. Heck, maybe they want to test it in court or something.
- pixl97 4y ago
  I'd say the vast majority of time when working with people programming/testing I hear the statement "Hmm, I didn't think of that".
  - deleted 4y ago
    [deleted]
- adamsmith143 4y ago
  You can of course just build a classifier to detect training set images and it would do very well even with subtly different images.
- RC_ITR 4y ago

SkyPuncher 4y ago

Seems to be no different than how people memorize "facts". I'm watching my daughter learn shapes right now. When I see a square, I call it a square and look for a matching "square" piece. It seems when my daughter sees a square, see is still looking at it's properties, labeling individual attributes (pointy thing here), then trying to find the same attributes on the corresponding piece. Eventually, my daughter will see a square enough that the concept will be re-enforced in her mind as a square. I wonder if a similar behavior is happening here. This specific image was presented enough times that it essentially "burned into" the AI. Rather than having to attempt to generate this image from scratch, the AI will simply pull out the canonical reference.

deleted 4y ago

[deleted]