6 ms·
>Until I can spin up a docker image capable of the same as OpenAI in hetzner for 30 bucks a month I do exactly this with https://github.com/nsarrazin/serge htt
by MacsHeadroom 3y ago
>Until I can spin up a docker image capable of the same as OpenAI in hetzner for 30 bucks a month
I do exactly this with https://github.com/nsarrazin/serge https://github.com/nsarrazin/serge
Hetzner will install any hardware you send them for $100. So you can send them a $200 P40 24GB to run 33B parameter GPU models at ChatGPT speeds without increasing your monthly cost.
- deanc 3y agoThat $200 card's price seems to have been hit hard by inflation in Finland [1] [1] https://www.proshop.fi/Naeytoenohjaimet/HP-Tesla-P40-24GB-GDDR5-RAM-Naeytoenohjaimet/2845785 https://www.proshop.fi/Naeytoenohjaimet/HP-Tesla-P40-24GB-GD...
- MacsHeadroom 3y ago180EUR https://www.ebay.de/sch/i.html?_from=R40&_nkw=nvidia%20p40&_sacat=0&_udlo=150&LH_PrefLoc=2&_udhi=230&rt=nc https://www.ebay.de/sch/i.html?_from=R40&_nkw=nvidia%20p40&_...