3 ms·
Nice - btw curious how much it costs to run Llama on this type of side project? I can imagine the cost goes to some reasonable amount when things like this get
by k2xl 2y ago
Nice - btw curious how much it costs to run Llama on this type of side project? I can imagine the cost goes to some reasonable amount when things like this get shared on HN. Does Fly.io give a flat price to use their inference? Curious how it works
- xena 2y agoI'd need to do the math on this, thankfully most of it is on a GPU kind that I don't normally use. I probably over-scaled so that I didn't need to think about the scaling issues, but I can tally it all up.
- k2xl 2y agothat would be great... i'm curious on the cost per million token as it compares to gpt-4o-mini