3 ms·
Interesting observation :) I guess this is the SQL query you have in mind that uses the LIKE operator: SELECT ChatGPT("Respond to the review with a solution
by jarulraj 3y ago
Interesting observation :)
I guess this is the SQL query you have in mind that uses the LIKE operator:
SELECT ChatGPT("Respond to the review with a solution to address the reviewer's concern", review)
FROM postgres_data.review_table
WHERE ChatGPT("Is the review positive or negative?", review) LIKE "%positive%"
AND location = “waffle house”;
From a query processing standpoint, both queries should have equivalent performance -- unless we build an index over the output of the ChatGPT query in EvaDB, in which case the former query would be faster than this one.
- remexre 3y agomm, no, not unless you're doing some LIKE-specific optimization (and even then, I think you'd want "positive%"). So like, at the end of all the decoders, the model gives you an output vector; you multiply this by your embeddings to get your token probabilities, then you sample from them to choose a token. Instead of sampling, you could just look at the probabilities for the tokens "positive" and "negative" and return whichever of those two is highest.
- jarulraj 3y agoYes, it should be "positive%". That's an interesting LLM-level optimization. Doesn't this token sampling optimization require using a locally-running model like Llama? I am presuming that OpenAI doesn't provide direct access to token probabilities in its API.