I had google assistant turned off but every now and then when using maps my phone would say out loud something like "Sorry I don't know how to help with <thing_

I had google assistant turned off but every now and then when using maps my phone would say out loud something like "Sorry I don't know how to help with <thing_we_just_said>". I couldn't find an easy way to turn it off in the app. All directions online lead to options that were already off. So like other users I had to restrict microphone access to google services entirely. I have no idea if it's still listening.