10 ms·

Reading these discussions with interest as I am in the process of making and training my own personal model using my 31+ archive of photography, pictures taken

by vr46 3y ago

Reading these discussions with interest as I am in the process of making and training my own personal model using my 31+ archive of photography, pictures taken and owned by be, which goes against this idea that models are not trained on data owned by the companies doing the training. While this is all for my own personal interest and use, how would the idea that the weights cannot be copyrighted affect my rights on the model if I were to release the whole thing for use?

londons_explore 3y ago
I'd be interested to know how your model performs if it is trained only on your own work. Assuming you haven't taken a photo a second for your entire life, then I suspect you'll struggle to make something even close to what's available publically, due to lack of training data.
- vr46 3y ago
  Obviously not, but the point isn't to make a public model, but to make something out of my own - my question was how does ownership work? At some point, somebody is going to be making something out of their own massive archive.
GaggiX 3y ago
How big is the archive? These models are typically trained on at least 100M images.
- vr46 3y ago
  There's around 1M images in there, I was wondering whether the labelling and object detection would be more important than the quantity.
  - GaggiX 3y ago
    1M images is probably enough to do something, this user for example trained a diffusion model from scratch using 1.5M images and a 3090: https://medium.com/@enryu9000/anifusion-diffusion-models-for-anime-pictures-138cf1af2cbe https://medium.com/@enryu9000/anifusion-diffusion-models-for..., the quality of course is not excellent but it's something. I suggest to train a 4x64x64 diffusion model using the new SD XL VAE (it's a really good f8 VAE so it can encode, for example, images from 3x512x512 to 4x64x64), if the images have captions then I suggest using a CLIP text encoder as it was already trained on image text pairs, it would probably be much easier to use by a diffusion model trained on only 1M images instead of other text encoders like T5 that have better text understanding but they have never seen an image.
  - lucubratory 3y ago
    Quantity is massively more important, to the point that you get much better results by using a larger dataset with machine generated labelling (or object detection if that's what you're building) than using a smaller one with even expert labelling. That said, if you've got a million photos you could probably do some pretty interesting things with very large scale fine tuning, or if you know many other people who have similar stockpiles of photos you may be able to get an entry-level dataset together if you all pool it.

by vr46 3y ago

londons_explore 3y ago

I'd be interested to know how your model performs if it is trained only on your own work. Assuming you haven't taken a photo a second for your entire life, then I suspect you'll struggle to make something even close to what's available publically, due to lack of training data.

vr46 3y ago
Obviously not, but the point isn't to make a public model, but to make something out of my own - my question was how does ownership work? At some point, somebody is going to be making something out of their own massive archive.

GaggiX 3y ago

How big is the archive? These models are typically trained on at least 100M images.

vr46 3y ago
There's around 1M images in there, I was wondering whether the labelling and object detection would be more important than the quantity.
- GaggiX 3y ago
  1M images is probably enough to do something, this user for example trained a diffusion model from scratch using 1.5M images and a 3090: https://medium.com/@enryu9000/anifusion-diffusion-models-for-anime-pictures-138cf1af2cbe https://medium.com/@enryu9000/anifusion-diffusion-models-for..., the quality of course is not excellent but it's something. I suggest to train a 4x64x64 diffusion model using the new SD XL VAE (it's a really good f8 VAE so it can encode, for example, images from 3x512x512 to 4x64x64), if the images have captions then I suggest using a CLIP text encoder as it was already trained on image text pairs, it would probably be much easier to use by a diffusion model trained on only 1M images instead of other text encoders like T5 that have better text understanding but they have never seen an image.
- lucubratory 3y ago
  Quantity is massively more important, to the point that you get much better results by using a larger dataset with machine generated labelling (or object detection if that's what you're building) than using a smaller one with even expert labelling. That said, if you've got a million photos you could probably do some pretty interesting things with very large scale fine tuning, or if you know many other people who have similar stockpiles of photos you may be able to get an entry-level dataset together if you all pool it.