3 ms·
OpenAI costs money and llama 2 i'm able to run on my GPU so at least for development purposes at the moment it's ""free"" for me to experiment with - mind you t
by boredumb 3y ago
OpenAI costs money and llama 2 i'm able to run on my GPU so at least for development purposes at the moment it's ""free"" for me to experiment with - mind you this is a side project with zero funding outside of myself, ymmv if you have access to funding to a point of making OpenAI tokens totally disposable.
- LewisVerstappen 3y agoAh okay, yeah I was curious how much you'd pay running it on a cloud server in a production-type scenario. Thanks
- boredumb 3y agoI have a vultr account I just logged into and checked, I could rent enough GPU memory to run llama 2 for ~180$ a month from my very quick look - so if I make more than 27000 requests with a 4k token payload I will break even - otherwise I could be using openAIs API and be making out better. EDIT** im sorry that was gpt3.5 - with gpt4 it would be 1000 requests before I broke even at 180$