4 ms·
I think the method of merging the pipelines via img2img should use controlnet. Possibly needing to be finetuned specifically for this, although existing control
by theaiquestion 3y ago
I think the method of merging the pipelines via img2img should use controlnet. Possibly needing to be finetuned specifically for this, although existing controlnet models might work fine for this.
This is exactly what you'd want to use controlnet for - mapping semantic information onto the perceived structure.
- tmabraham 3y agoYes we've been looking into ControlNet as well, and I think there is one recent fMRI-to-image paper that also has tried ControlNet. Maybe we'll use ControlNet in MindEye v2 :)
- atom_101 3y agoYes controlnet will be used in the next version. For this one we couldn't get it working in time.