3 ms·
> it is non-trivial to predict which searches GPT will be good for Since pricing is roughly proportional to response length (and maybe prompt length?), it seem
by chrisshroba 4y ago
> it is non-trivial to predict which searches GPT will be good for
Since pricing is roughly proportional to response length (and maybe prompt length?), it seems like ChatGPT could use itself to determine if the search is a good fit or not. Give it a prompt like "I want to know if you are likely to generate more immediately useful and actionable results than my web search. The query I am searching is <query>. Should I run you on this query?"
I tried running that on some sample queries to see its output, and compiled its responses in this table (note the last one is incorrect in my opinion):
| Query | Yes or No |
|------------------------------------|-----------|
| "starbucks near me" | No |
| "twitter" | No |
| "python requests get json" | Yes |
| "peach and mango cocktail recipe" | Yes |
| "hn" | No |
| "image of versaille" | Yes |
Btw, incredibly, ChatGPT actually made that table for me when I said "Please compile all the queries I just asked you about into an ascii table, where the first column is my query, and the second column is your Yes or No answer." The table it printed was correct but formatted as an HTML table, so I asked it to put it in a code block and I got almost exactly what I pasted above (just had to remove two extraneous spaces that made the lines not line up)
- freediver 4y agoYep, this is one of the amazing things about it. But using ChatGPT to determine whether a query is good for ChatGPT answer does not address the second point I made, one about typical latency requirements (it makes it worse). Normally you want your search results in about 500ms. Using ChatGPT to first figure out if the query is good will take 1-3 seconds then using it again another 1-3 seconds. So we are talking about results in about 4 seconds. Plus it is not 1c per query now it is more likely 1.5c per query as you introduced an extra prompt.