3 ms·
Well, if you have an app that's a thin structure on top of GPT-4, adding your own pipe between your client and OpenAI could add a lot to the cost and complexity
by joe_the_user 4y ago
Well, if you have an app that's a thin structure on top of GPT-4, adding your own pipe between your client and OpenAI could add a lot to the cost and complexity of the app. Which is to say it's not surprising that people don't do it.
The genius and the craziness of GPT-4 is you can make whole app with a prompt like "now you're a clown painting custom faces on kids based on their favorite animals" and some glue-code. Needing to add a 3 layer network infrastructure with isn't appealing I'd imagine.
- mousetree 4y agoCould be as simple and cost-efficient as a Cloudflare worker that adds your key and passes the query along
- lucakiebel 4y agoOr a 5€/mo Hetzner server running Nginx, though Cloudflares free offering is very generous
- jeffy90 4y agoYou'd need some kind of authentication as well.
- pxx 4y agoErrrr, I assume your app can pass said authentication? If so, then it's meaningless; that's again a secret not under your control.
- shri_krishna 4y agoYou have to use Cloudflare Workers in Unbounded mode to do this (especially if you are streaming using EventSource or WebSockets). Bundled mode won't cut it as it closes connection after 50ms of Javascript execution.
- orf 4y agoThat’s 50ms of CPU execution, which does not include waiting for IO.
- de6u99er 4y agoNot only that, but it makes not much difference if they call the GPT Api directly or through a proxy. Only thing that eould really help is having users register and authenticate through the proxy.
- ceejayoz 4y agoThe proxy approach at least lets you rate limit by IP, limit the length of the strings (and thus the token cost), etc. The API key may also grant access to other models, administrative IPs, etc. you don't want people using. Far, far, far better than nothing.