3 ms·
Most developers avoid GPU because of pricing. It’s simply too expensive to run 24/7 and there is the overhead of managing / bootstrapping instances loading larg
by zacksiri 2y ago
Most developers avoid GPU because of pricing. It’s simply too expensive to run 24/7 and there is the overhead of managing / bootstrapping instances loading large models to do intermittent instances. That’s the gist of it I think.
Unless you have constant load that justify 24/7 deployments most devs will just use an API. Or find solutions that doesn’t need you to pay > $1 / hour.