4 ms·
> Developers can run inference on Llama 3.1 405B on their own infra at roughly 50% the cost of using closed models like GPT-4o Does anyone have details on exac
by popcorncowboy 2y ago
> Developers can run inference on Llama 3.1 405B on their own infra at roughly 50% the cost of using closed models like GPT-4o
Does anyone have details on exactly what this means or where/how this metric gets derived?
- rohansood15 2y agoI am guessing these are prices on services like AWS Bedrock (their post is down right now).
- PlattypusRex 2y agoa big chunk of that is probably the fact that you don't need to pay someone who is trying to make a profit by running inference off-premises.