I think the first question to ask is what the contract says about your separation from this client. It's great that you feel a moral anxiety about leaving them

I think the first question to ask is what the contract says about your separation from this client. It's great that you feel a moral anxiety about leaving them but remember that it's a business relationship - that means each side trades something for another thing they deem to be more valuable. If either side doesn't feel this way (and clearly you don't) then things have to change. Your job is to control the change. Even though you're not an "employee", I don't see why you can't effectively "resign". Give some period of notice and stick to it. Also beware that this might lead to a renegotiation of your current deal. Have you made your points known to the client? If not then you're being a bit unfair.