There are still distributions that do not use systemd though, e.g. two that come to my mind are Slackware if you are more conservative or Void if you prefer a r

There are still distributions that do not use systemd though, e.g. two that come to my mind are Slackware if you are more conservative or Void if you prefer a rolling distro. And TBH IMO the choice of window system affects way more the day-to-day operations of a desktop oriented distribution than the init system ever did, so i expect slightly more friction about removing Xorg (as a sidenote the server is Xorg, X11 is the protocol and another server - most likely a fork of Xorg - can continue providing it if Xorg shuts down).