Eh, "remove empty page" is not "live text editing". I have seen tools struggle with seemingly simple operations, like "take page 1,4 from A and pages 4,6 from B

Eh, "remove empty page" is not "live text editing". I have seen tools struggle with seemingly simple operations, like "take page 1,4 from A and pages 4,6 from B, scale to fit paper size and combine into one file". And those are not some exotic requirements, those are one of the most basic operations people want to do with multi-page documents. Also, did you want to say JPEG isn't suitable for collaborative image work (because it's lossy?). As PNG is a pretty fine exchange format if two people, who prefer different graphical editors, want to work on the same document.