4 ms·
I have a vultr account I just logged into and checked, I could rent enough GPU memory to run llama 2 for ~180$ a month from my very quick look - so if I make mo
by boredumb 3y ago
I have a vultr account I just logged into and checked, I could rent enough GPU memory to run llama 2 for ~180$ a month from my very quick look - so if I make more than 27000 requests with a 4k token payload I will break even - otherwise I could be using openAIs API and be making out better.
EDIT** im sorry that was gpt3.5 - with gpt4 it would be 1000 requests before I broke even at 180$