I started out using $HOME/bin, but a fair amount of stuff assumes a /usr- or /usr/local-style folder structure when doing make install, so I've settled on using

I started out using $HOME/bin, but a fair amount of stuff assumes a /usr- or /usr/local-style folder structure when doing make install, so I've settled on using $HOME/usr/bin instead, so that programs can create $HOME/usr/include and $HOME/usr/share and whatever, without trampling on stuff in my home folder. Can't remember the last time I had a problem arranging this. If using autotools, which covers 95+% of stuff, it's usually a question of something like "./configure --prefix=$HOME/usr". (If I want to share stuff between users, /usr/local/ is of course a better place. macOS is a bit more restrictive, so I have a separate user for this, whose /usr folder is readable by everybody.)