The majority of water is used in ways that none of it would be drank. Even just looking at homes, only about 20% of water in a typical home is used in the fauce

The majority of water is used in ways that none of it would be drank. Even just looking at homes, only about 20% of water in a typical home is used in the faucet (and I expect that's primarily not for drinking). You'd almost need to set up an entirely separate distribution system for it to be more efficient than doing it in the home.