3 ms·
Thanks everybody for trying this out. I just looked at our logs and we're doing >2k requests per minute to OpenAI right now. Free users only get partial search
by philip1209 2y ago
Thanks everybody for trying this out. I just looked at our logs and we're doing >2k requests per minute to OpenAI right now.
Free users only get partial search results. If there are any you want to see run to completion, reply here or email me and I'll mark it to run to completion. (The code PRODUCTHUNT is also available this week for a free month of access).
- philip1209 2y agoUpdate: >5k requests per minute to OpenAI right now
- esafak 2y agoDon't forget to rate limit every end point.
- philip1209 2y agoYup, we have some decent safeguards in place. It’s technically difficult to allow logged out users to try the product, but I think people appreciate it.
- Palmik 2y agoCongrats on the traffic, I hope you get a high conversion rate :) I assume you do more than one GPT call per search, e.g. rewriting user's query into multiple search queries, summarization of the results, etc.?
- philip1209 2y agoIt's complex. We make a lot of GPT calls. Lots of them. And, for lots of different purposes.
- jsemrau 2y agoHow are you handling the unit costs, rate limits, and model risk?
- philip1209 2y agoThat’s a deep question. Under the hood we use some LLM gateways like Helicone and https://usevelvet.com https://usevelvet.com to track cost per request and optimize it.