I always thought the point with memory segments was that you could load a program verbatim without adjusting any pointers and do tricks like Terminate and Stay

I always thought the point with memory segments was that you could load a program verbatim without adjusting any pointers and do tricks like Terminate and Stay Resident and executing one program from another. E.g. Command.com and edlin.com don't need to know where they're loaded when they're both in memory, they just start at CS:0100.