4 ms·
Please do! Adding in a Segment Anything Model UI to the app would be great. You can then click on the items you want removed and use the segments as inpainting
by iandanforth 3y ago
Please do! Adding in a Segment Anything Model UI to the app would be great. You can then click on the items you want removed and use the segments as inpainting masks for the generative model.
- dwallin 3y agoLeveraging multiple models here (Segment Anything is a good candidate) seems like a must to build a defensible product. Using just SD with basic control net and maybe some additional finetuning makes it very cloneable. If you leverage other models you can do things like identify what elements of the room must be preserved (walls, windows) and what can be safely removed or edited. This could allow you to generate a simulated image of the room without furniture as a base. Derive a 3d projection to allow the user to place basic 3d furniture stubs within the space. Then use those stubs to create subject masks with the furniture-less base to render a final room.