4 ms·
Ah okay, yeah I was curious how much you'd pay running it on a cloud server in a production-type scenario. Thanks
by LewisVerstappen 3y ago
Ah okay, yeah I was curious how much you'd pay running it on a cloud server in a production-type scenario. Thanks
- boredumb 3y agoI have a vultr account I just logged into and checked, I could rent enough GPU memory to run llama 2 for ~180$ a month from my very quick look - so if I make more than 27000 requests with a 4k token payload I will break even - otherwise I could be using openAIs API and be making out better. EDIT** im sorry that was gpt3.5 - with gpt4 it would be 1000 requests before I broke even at 180$