I struggled through the same thing almost 15 years ago and finally found a middle ground. We had a team of three co-located with the customer and then we arran

I struggled through the same thing almost 15 years ago and finally found a middle ground. We had a team of three co-located with the customer and then we arranged for everyone to work remotely one day a week, barring Monday or Friday (for appearances). Tuesday was my day. After a few successful months of that, we expanded it out to 2 days each. We would have gone to three but the contract wrapped. But it gave us a small space to experiment and see what would work and slowly convince the customer that it would be okay.