2 ms·
Training these models is much much cheaper than you think it is, and there’s good data for it already. Either use a CC0 set like Wikimedia/Flickr and throw in
by astrange 4y ago
Training these models is much much cheaper than you think it is, and there’s good data for it already.
Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.
- trention 4y agoYou are getting ridiculous now. Training this on "Taobao product photos" will lead to a useless model that is unable to produce practically all of the "cool" demos posted here in the last week.
- angusturner 4y agoA few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website. As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible. Training from scratch is about $500k still, but it will only get cheaper and easier.
- trention 4y agoThis doesn't contradict anything I have written. The average technical user will be unable to train this exact model (not to mention the supposed future more powerful ones) in their basement in this decade.
- angusturner 4y agoThat just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall. But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch. Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit. Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).