This may be good if you know all your rect sizes in advance. But it seems like it will quickly fall apart if you have rectangles that need to change their size

This may be good if you know all your rect sizes in advance. But it seems like it will quickly fall apart if you have rectangles that need to change their size based on content, especially if that content is multiple levels deep. Say you have a floating toolbar the width of two buttons, which themselves are the width of their text contents. You're back to square one — you have to measure the text of each button, then measure both buttons and finally you have the size of the toolbar.