Features is indeed the answer, and all of the potential issues are manageable. I finished up a project earlier this year that has more than 300 modules active

Features is indeed the answer, and all of the potential issues are manageable. I finished up a project earlier this year that has more than 300 modules active in a custom install profile, with 10MB of features modules stuck in there. We committed/contributed to a few fixes in core and Features to speed it up greatly. That's out in the wild on major sites and doing fine. We didn't have any terrible admin screen issues, or indeed any terrible issues beyond occasional painful version control collisions.