I did something similar a decade ago (well without the UNIX socket bit, but just a separate sshd listening on localhost only and also no need to deal with SCM_R

I did something similar a decade ago (well without the UNIX socket bit, but just a separate sshd listening on localhost only and also no need to deal with SCM_RIGHTS). Nothing good or bad came out of it. I simply got bored and didn't bother porting this setup to the next machine.