Yep, this is one of the amazing things about it. But using ChatGPT to determine whether a query is good for ChatGPT answer does not address the second point I m

Yep, this is one of the amazing things about it. But using ChatGPT to determine whether a query is good for ChatGPT answer does not address the second point I made, one about typical latency requirements (it makes it worse). Normally you want your search results in about 500ms. Using ChatGPT to first figure out if the query is good will take 1-3 seconds then using it again another 1-3 seconds. So we are talking about results in about 4 seconds. Plus it is not 1c per query now it is more likely 1.5c per query as you introduced an extra prompt.