There are already various servers you can set up at home or on a personal server based on CardDAV, which is an open standard for exactly this purpose. I set one

There are already various servers you can set up at home or on a personal server based on CardDAV, which is an open standard for exactly this purpose. I set one up for myself in a day, and I can easily access it on my phone and, from a web UI, on my computers, in my email client, etc. It's fairly widely supported and you can use it with various calendar apps and programs, but unfortunately the standard is a bit awkward and it's still not as widely supported as things like Google Calendar.