Despite the sword and sorcery veneer this is actually the premise of several of CJ Cherryh's works, for example : https://en.wikipedia.org/wiki/Exile's_Gate htt

Despite the sword and sorcery veneer this is actually the premise of several of CJ Cherryh's works, for example : https://en.wikipedia.org/wiki/Exile's_Gate https://en.wikipedia.org/wiki/Exile's_Gate