7 ms·
the price is pretty low: https://www.together.ai/pricing https://www.together.ai/pricing 4B MODEL, PRICE 1K TOKENS: $0.0001 register with an email, test acc
by nittanymount 3y ago
the price is pretty low: https://www.together.ai/pricing https://www.together.ai/pricing
4B MODEL, PRICE 1K TOKENS: $0.0001
register with an email, test account has 25$ credit,
python API as well,
the model is smaller, but good to have some fun with the API integrated with other systems, haha...
- valine 3y agoNot really fair to compare a 4B model and a 1.7T model. Per flop the 4B model here is slightly more expensive than GPT4.
- The_Contrarian 3y agoThe theory that GPT-4 is 1.7T also posits that GPT-4 is composed of eight 220B experts, meaning once you've loaded the model, inference costs aren't akin to a 1.7T model, but instead to a 220B model. If prices scale linearly, we reach 0.03 / 1K tokens (lower end of GPT-4's price range) at about 0.0001 * 300, or about 1.2T parameters (and that's dense - no MoE here).
- valine 3y agoI guess the question is how the mix of experts works. Do they predict one token from each model? If so you’re still doing 1.7T worth of computation.
- erikaww 3y agoclause 2.1 is $8/million tokens for prompt or .008/1k tokens. interesting