3 ms·
Can you go into a bit more detail? What architecture did you use? Is the month training time really just training with mini batches with a constant learning rat
by hackater 4y ago
Can you go into a bit more detail?
What architecture did you use? Is the month training time really just training with mini batches with a constant learning rate? Or are these many failed attempts until you trained a successful model for a few days in the end?
I particularly interested in the image generation part (the DDPM/SGM)
- Jack000 4y agoYeah I did have a few false starts. Total time is more like 3 months vs 1 month for the final model. For small scale training I found it’s necessary to use a long lr warmup period, followed by constant lr. There’s code on my GitHub (glid3) edit: The architecture is identical to SD except I trained on 256px images with cosine noise schedule instead of linear. Using the cosine schedule makes the unet converge faster but can overfit if overtrained. edit 2: Just tried it again and my model is also pretty bad at hands actually. It does get lucky once in a while though.
- rrobukef 4y agoI keep wondering if using not only statistical noise but also deformations would help with the generation of deformable things - say human hands.
- dmingod666 4y agoF222 does a little more coherent anatomy..not surprising given its background