If the same code was all aggregated in a single large crate, it would still be compiled as a single unit, so I don't see how it would affect compilation times.

If the same code was all aggregated in a single large crate, it would still be compiled as a single unit, so I don't see how it would affect compilation times. And, of course, that is orthogonal to security issues. That pip/npm/cargo model doesn't play well with distro package managers is not a new thing. I think that eventually the distro packagers will have to bite the bullet and just start packaging apps that need any of these as bundles (which, yes, means that they need to be serviced separately for security updates; but there's no reason why the metadata used to decide when that is necessary should be the same as install-time package dependency graph).