3 ms·
This is a really compelling experience Curious-- are you "invoking" the model for every user or caching some of the results? I've been obsessed w/ using GPT3
by valgaze 5y ago
This is a really compelling experience
Curious-- are you "invoking" the model for every user or caching some of the results?
I've been obsessed w/ using GPT3 for "personas" in conversations (for the agent to take on a character or follow instructions), but didn't seem feasible to make an API call (esp to Davinci) for every user
- arcastroe 5y agoI agree the API is a bit expensive. I'm fortunate that this particular use-case is fit for caching. It caches every result and only invokes OpenAPI upon clicking the "Generate More" button. The response is then saved for all future users of the site. So far everything on the page has been generated for under $10. But I can see how it would become prohibitively expensive if generating unique text for each user.