4 ms·

I turn off as many settings for Assistant as I can, and disallow the Microphone permission to the "Google" app. That should help you out.