I used Gentoo as my main desktop OS for a year. The system will not get much faster because of the CFLAGS one uses (this is especially true for x86_64), but you

I used Gentoo as my main desktop OS for a year. The system will not get much faster because of the CFLAGS one uses (this is especially true for x86_64), but you can customize the packages you build to contain exactly the functionality you need (through the use of USE flags). In the end, I switched over to Arch Linux (got tired of waiting for compiles), then switched to Debian testing (needed loads of LaTeX packages not found in Arch Linux), before I switched back to Arch a couple of weeks ago.