4 ms·
related: I had an idea to make a CLIP paint.NET image editing plugin. You would make a new layer, add some green ('towards') and red ('away from') splotches to
by pizza 5y ago
related: I had an idea to make a CLIP paint.NET image editing plugin. You would make a new layer, add some green ('towards') and red ('away from') splotches to the layer, and give a prompt to CLIP. Wherever there was green, CLIP would 'nudge' the image along the direction of the gradients towards the semantic goal, keep it constant where there was nothing, and nudge it in the opposite direction where there was red. And you could do this over and over again, w/ different CLIP prompts all the while editing the semantic masks as you like. The end result would be a feedback process to use CLIP + feedback as an extremely creative and controllable image manipulation tool. But then I realized that I didn't know how to make a paint.NET plugin :) so here is a free idea for anybody who does
- yeldarb 5y agoIt doesn't use CLIP but check out NVIDIA's Canvas[1] (powered by GauGAN[2] which has a pretty poor UI but is usable in your browser). [1] https://www.nvidia.com/en-us/studio/canvas/ https://www.nvidia.com/en-us/studio/canvas/ [2] http://gaugan.org/gaugan2/ http://gaugan.org/gaugan2/
- pizza 5y agoYep, seen Canvas - probably where I Picasso'd (read: stole :^) ) the original idea! The nice thing about making it a PDN plugin based on CLIP is that it could plug into the existing PDN plugin system, PDN is pretty solid image editing software, and also CLIP could make it extremely versatile - no more limitation to primarily landscapes - including the fact that there are newer and vastly better CLIP tools coming out, e.g. diffusion model based CLIP - but I don't think I had actually seen the GauGAN UI before, thank you!