It's not a bad idea but it definitely has downsides. For example, if your GV number is on the same Google account as your email, and you use the GV number for 2

It's not a bad idea but it definitely has downsides. For example, if your GV number is on the same Google account as your email, and you use the GV number for 2fa, then you've turned two points of failure into a single point of failure. The point of 2fa is to use "something you have access to" and "something you have" (e.g. a device). By using the same account for both steps of 2fa, you defeat the purpose of it, in the event your Google account is compromised. Whether that is safer than the status quo of a 2fa SMS number at a wireless carrier is up for you to decide.