Leveraging multiple models here (Segment Anything is a good candidate) seems like a must to build a defensible product. Using just SD with basic control net and

Leveraging multiple models here (Segment Anything is a good candidate) seems like a must to build a defensible product. Using just SD with basic control net and maybe some additional finetuning makes it very cloneable. If you leverage other models you can do things like identify what elements of the room must be preserved (walls, windows) and what can be safely removed or edited. This could allow you to generate a simulated image of the room without furniture as a base. Derive a 3d projection to allow the user to place basic 3d furniture stubs within the space. Then use those stubs to create subject masks with the furniture-less base to render a final room.