There's a big difference between something that's possible with some effort (i.e. compile the Linux kernel from source, compare the binaries with what ships wit

There's a big difference between something that's possible with some effort (i.e. compile the Linux kernel from source, compare the binaries with what ships with your distro), and it doesn't need to be you personally that does it. If a distro has 100k users, only 1 of them has to discover and make a scandal out of it. Projects like Debian take it even further and make a promise that it should be easy to reproduce the builds of their packages. That's literally impossible with any closed-source server-side software. You can't even inspect the binary as you can with a client.