One of my biggest pain points in setting up an unfederated homeserver for my friend group was trying to figure out how to not have so much access as the server

One of my biggest pain points in setting up an unfederated homeserver for my friend group was trying to figure out how to not have so much access as the server admin. I am nowhere near an expert at cybersecurity or cryptography or sysadmin things, but I had a strong sense of "I shouldn't be allowed to do this" - for example, when users had difficulty verifying their sessions, I was able to simply click a link in Element to "manually verify" the user on behalf of the entire encrypted group. This was several years ago and the verification flow seems to have improved since then, but that gave me some misgivings.