3 ms·
You can rent GPUs by the hour and there are (limited) serverless options too. So this question makes a gigantic difference in cost. >of course if I'm launching
by Havoc 3y ago
You can rent GPUs by the hour and there are (limited) serverless options too. So this question makes a gigantic difference in cost.
>of course if I'm launching a public product it should be available 24/7.
Generally yes, but not a given. In some context like data processing you can accumulate tasks and run them in a batch.
e.g. OpenAI just launched batches...50% discount but you don't get instant results, you get them somewhere in next 24h
- _akhe 3y agoAhh got it, so not for a real-time consumer use cases then. Typically when I think of a user using an AI product they're hitting the LLM a lot and expecting results on their screen right then, but I guess there are a lot of back office and data processing use cases.