I've heard Jews and Christians having issues with their own buildings, and other buildings being called 'temple', so all I know is that's not the silver bullet

I've heard Jews and Christians having issues with their own buildings, and other buildings being called 'temple', so all I know is that's not the silver bullet if you're trying to do a word swap. I think the right way to go is to generalize so we're not reading more into it than we know, "religious building" or "religious site", "religious area", or maybe "ceremonial building" if you're not sure the nature of the use. On the other hand, if you are certain about what it was, then you can be specific and it won't be wrong or need correcting.