3 ms·
Not sure what you run this on and why its $50/day but sounds like a lot. Convert your server cost to a pay as you go model. You are currently paying even for t
by irfn 3y ago
Not sure what you run this on and why its $50/day but sounds like a lot.
Convert your server cost to a pay as you go model. You are currently paying even for time when your server is idle.
Containerise your app and run on something like AWS Lambda which has a pay for what you use model.
- s3 is cheap for storing your model data and as long as you are just reading that should be fine.
- AWS Lambda has 1 million free invocations per month in free tier.
- Post that you pay $0.60 per 1 million requests
- AWS stack is just an example, you can easily use GCP Cloud run or any other equivalent service.
- gbasin 3y agothe primary cost would be the OpenAI GPT-3.5-turbo API, which is ~ $0.002 per 1k tokens
- laratied 3y agoI have made a chatbot like this and I am not a software engineer. The whole thing is trivial to make. The system prompt is trivial. I would even say it was harder to make a geocities page in the 90s than to make a chatbot like this. 3.5 turbo is great to play with but at scale it is brutal cost wise. The real issue though is anyone who already has a working 3.5 turbo chatbot can knock this off in the next 5 minutes by just changing the system prompt. I think this is the future though. The labor is pretty much trivial, the capital for tokens at scale is what matters.
- mejutoco 3y agoMy guess is the 50$ come from the open api charges, not from the laravel app. Can you cache some of the interactions and serve the cached version for the most common ones?