Even outside of the Play Store, an APK cannot be updated by another one using a different signature. It's a verification made by the OS itself. F-Droid themsel

Even outside of the Play Store, an APK cannot be updated by another one using a different signature. It's a verification made by the OS itself. F-Droid themselves compile and signs the binaries offered by their repository, which everyone seems to be fine with. I believe they're also working towards reproducible builds to let others independently verify the compilation matches the source code.