I dont want to use the voice assistant on my pixel but the function is so baked in that if I connect a headset, despite never setting the assistant up, it will

I dont want to use the voice assistant on my pixel but the function is so baked in that if I connect a headset, despite never setting the assistant up, it will interupt any music or playback after 15 seconds to tell me the assistant is ready now and there is no way to make that message go away or resume my media playing.