2 ms·
No... I mean LLM's use a GPU. Hence why the cost is different. Changing Google Translate to use an LLM would become much more expensive for Google. This is wh
by ImHereToVote 3y ago
No... I mean LLM's use a GPU. Hence why the cost is different.
Changing Google Translate to use an LLM would become much more expensive for Google.
This is why you see a different cost structure for using "AI". LLM's can't realistically use the CPU for anything serious.
- trashtester 3y agoYou have it backwards. For any given amount of compute needed, GPU's are a lot cheaper than CPU's. But GPU's can't run all the code you can run on a CPU. The reason models that use GPU's cost more to run, is that they tend to be A LOT more compute intensive. However, if you run the inference for the same models on CPU, they will be both much more expensive than on GPU's (or on specialized tensor silicon) and also slower.