6 ms·
Something I haven’t seen anyone talking about with these huge models: how do future models get trained when more content online is model generated to start with
by nharada 4y ago
Something I haven’t seen anyone talking about with these huge models: how do future models get trained when more content online is model generated to start with? Presumably you don’t wanna train a model on autogenerated images or text, but you can’t necessarily know which is which.
- cpach 4y agoMakes me think of Ouroboros https://en.m.wikipedia.org/wiki/Ouroboros https://en.m.wikipedia.org/wiki/Ouroboros
- nharada 4y agoReminds me of https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
- espadrine 4y agoIn this situation, the low-background steel is the MS-COCO dataset, associated with the Fréchet inception distance computed by comparing the statistical divergence between the high-level vector outputs of passing MS-COCO images through Google’s InceptionV3 classifier, and passing DALL-E images (or its competitors) through it. For now at least, there is a detectable difference in variety.
- zitterbewegung 4y agoThis should be a step in cleaning your data to begin with. If you don't know the providence of your data then you shouldn't be even training with it. Getting humans to refine your data is the best solution right now and many companies and researches go with this approach.
- nharada 4y agoBut how would you know? A random string of text or an image with the watermark removed is going to be very hard to distinguish generated from human written.
- FrenchDevRemote 4y agoYou can't use humans to manually refine a dataset on the scale of GPT-3 or DALL-E Clip was trained on 400,000,000 images, GPT is roughly 180B tokens, at 1-2 tokens per word, that's 120,000,000,000 words.
- pshc 4y agoAt least cleaning it up is an embarrassingly parallel problem, so if you had the resources to throw incentives at millions of casual gamers, you might make a nice dent on Clip.
- zanderwohl 4y agoAlternatively, making a captcha where half the data is unlabeled, and half is labeled, forcing users to categorize data for you as they log into accounts.
- jazzyjackson 4y agos/ide/ena
- sailingparrot 4y ago> Getting humans to refine your data is the best solution right now Source ? All those big models are trained with data for which the source is not known or vetted. The amount of data needed is not human-refinable. For example for language models we train mostly on subsets of CommonCrawl + other things. CommonCrawl data is “cleaned” by filtering out known bad sources and with some heuristics such as ratio of text to other content, length of sentences etc. The final result is a not too dirty but not clean huge pile of data that comes from millions of sources that no human as vetted and that no one in the team using the data knows about. The same applies to large images dataset, e.g. Laon 400m that also comes from CommonCrawl and is not curated.
- Jleagle 4y agoThe images i have created all have a watermark.. This is at least one way to filter out most images, by the same AI at least.
- Gigachad 4y agoIt’s trivial to remove the watermark and other tools put no watermark on.
- mikeyouse 4y agoThis precise thing is causing a funny problem in specialty areas. People are using e.g. Google Lens to identify plants, birds and insects, which sometimes returns wrong answers e.g. say it sees a picture of a Summer Tanager and calls it a Cardinal. If the people then post "Saw this Cardinal" and the model picks up that picture/post and incorporates it into its training set, it's just reinforcing the wrong identification..
- scarmig 4y agoThat's not really a new problem, though. At one point someone got some bad training data about an old Incan town, the misidentification spread, and nowadays we train new human models to call it Macchu Picchu.
- vanillaicesquad 4y agoThe difference between the name of an old Incan town and a modern time plant identification mistake is that maybe the plant is poisonous. Made with gpt3
- Pxtl 4y agoThen that's a cardinal now.
- bobbylarrybobby 4y agohttps://xkcd.com/978/ https://xkcd.com/978/
- samstave 4y agoImagine when there is an AI that is monitoring content creation and keeping tabs of original sources....
- goolulusaurs 4y agoIt's a cybernetic feedback system. Dalle is used to create new images, the images that people find most interesting and noteworthy get shared online, and reincorporated into the training data, but now filtered through human desire.
- deleted 4y ago[deleted]
- jmartrican 4y agoI wonder if human artists can demand that their work not be used for modelling. So as the robots are stuck using older styles for their creations, the humans will keep creating new styles of art.
- qualudeheart 4y agoThey’ll ignore it either way.
- naillo 4y agoOne interesting comment about this is that some models actually benefit from being fed their own output. Alphafold for instance was fed with its own 'high likelihood' outputs (as demis hassabis described in his lex friedman interview).
- blfr 4y agoTraining on auto generated images collected off the Internet is gonna be fine for a while since the images surfacing will be curated (ie. selected as good/interesting/valuable) still mostly by humans.
- can16358p 4y agoI think with the terms requiring explicitly telling which images/parts were generated, they could be filtered out and prevent a feedback loop of "generated in/generated out" images. I'm sure there will be some illegal/against terms of use cases there but the majority should represent fair use.
- gwern 4y agoMy discussion of this issue (which actually comes up in like every DALL-E 2 discussion on HN): https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-e-2-can-and-cannot-do?commentId=CWKFyJYfgoZfP9955 https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-...