4 ms·
Hmmm, the Github repo suggests that you might be able to run the 65B model on a single A100 80gb card. At the moment, the spot price on Google cloud for this ca
by rnosov 4y ago
Hmmm, the Github repo suggests that you might be able to run the 65B model on a single A100 80gb card. At the moment, the spot price on Google cloud for this card is $1.25/hour which makes it not so crazy expensive...
- nabla9 4y ago$1.25/hour is roughly a year of GPU time until it exceeds the price of A100 80GB card.
- metadat 4y agoI think OP meant that $1.25/hr makes this accessible for people try it out themselves cost effectively, without having to spend thousands or tens of thousands up front to obtain a capable hardware rig. Obviously $1.25/hr 24/7 does add up quickly, after one month the bill would come to $900.
- deleted 4y ago[deleted]