5 ms·
150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rate
by nostrebored 1mo ago
150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool.
Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information.
```
Billing access restricted
Self-serve billing is not available on Enterprise accounts. Please contact your team for further questions.
```
We have no team (they removed themself from our slack channel after we talked about rate limits). Perplexingly, none of this even shows up in the request, which gives:
```
{"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"}
```
When the error is really about billing.
I always want to like Cerebras, but I get the vibe that as a tokens in tokens out consumer you are not valued at all.
- olivermuty 1mo agoCerebras the tech is awesome, cerebras the company is a trainwreck
- dd8601fn 1mo agoIs this the chatjimmy asic approach with a bigger model?
- ericd 1mo agoNo, the asic could only ever run one model/set of weights, no updates possible, ever. These are general purpose processors that can have their models updated. But the chips are enormous, with a substantial amount of on-die memory alongside the execution units, for a relatively insane amount of memory bandwidth.
- vel0city 1mo agoI thought from what I read about the Taalas approach, the model architecture and overall size couldn't be changed, but model weight values could be updated after for further tuning. Not as flexible as Cerebras though. And I'd love for someone who knows more to clue me in to the truth.
- ericd 1mo agoNah, Taalas was putting the weights into silicon as a mask ROM. Their demo chip was hardwired to serve Llama 3.1 8B, and could never be updated. New models, even new versions without any architectural/size changes meant new tape outs. But in exchange, you get insane speed and great energy efficiency. I could see it being a great approach for basic "good enough" models. They may have had a little flexibility by supporting finetuning via LoRAs.
- liamwire 1mo agoTo qualify "insane speed" for anyone unaware, think a 10x improvement over even Cerebras. On the order of ~15,000 tokens/s. Not saying their approach scales well enough to keep pace with the various frontiers, but using their demo alone feels like a paradigm shift.
- runako 29d agoThis seems like it would be an altogether different experience than the common experience of using an LLM, which is characterized by the person spending a lot of time waiting on the machine.
- ericd 29d agoYeah, it'd be a lot easier to maintain flow, less need to work on more than one session at once, etc. And then tool calls would be the limiting factor, especially network access. I hope AMD keeps the project moving forward post acquisition.
- ericd 29d agoYeah, I'm guessing this isn't unique, but I remember the first time I used ChatJimmy, I missed the fact that it had responded because I was still hitting the enter key, and getting ready to see tokens stream in, but they were already all sitting there, and I'd missed registering the visual diff somehow.
- Tuna-Fish 1mo ago
- cute_boi 1mo agoi hope groq wins if they start doing such things with consumer.
- deleted 1mo ago[deleted]
- 0xbadcafebee 1mo agoYeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
- LoganDark 1mo agoGPUs can't reach these speeds. You could build a supercomputing cluster and still not reach these speeds.
- gpugreg 1mo agoMiMo-V2.5-Pro-UltraSpeed gets pretty close with over 1000 TPS on 8x B200. It has 1.02T total parameters and 42B active, compared to 27B total/active for Qwen3.8-27B. Also, B300 are out now. I think 1500 TPS for Qwen3.8-27B should be doable.
- LoganDark 1mo agoThat model uses a lot of tricks to achieve 1000 t/s. I would not use raw parameter counts alone for such comparisons, in general.
- nostrebored 29d agoCeleris reaches ~50% of the speed on commodity hardware. celeris-magnus-1 is based on qwen3.8-27b. Maybe we will get there without custom silicon!
- collin 1mo agoThis was my experience a year ago on some other model they could run super fast. Routine coding tasks would hit the per-minute token limits. Just the math there... 150k TPM... and 15k TPS means... you can run for 10 seconds every minute? The basic math boggles the mind.
- baegi 1mo agoNot sure how the rate limiting works, but it's 1.5k TPS, not 15k, so you could run it for 100s/min, which seems good enough to me
- nostrebored 1mo agoiirc input (uncached) goes towards the limit as well
- fc417fc802 1mo agoWhat's the tok/s when they process input?
- fc417fc802 1mo agoIt seems you forgot to account for the fact that cerebras uses a baker's minute which is 144 seconds instead of 60. (Seriously though what's the supposed issue here?)
- RussianCow 1mo agoThe issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads.
- collin 1mo agoah, yes, that seems right I was using it quite a while back, different model, different quotas, but for coding tasks it routinely hit quotas which made it quite difficult to actually use. 100s/min seems pretty poor actually with sub-agents etc.
- ricardobeat 1mo agoWhat kind of coding tasks would you expect to hit that limit? In my setup, on a very large codebase, it takes each agent 3-4 minutes at minimum to go past 100k tokens. (note it's 150k uncached tokens, the total limit is 450k/min)
- nostrebored 1mo agoin my last tests with cerebras for coding tasks, most large tasks or anything greenfield would hit token limits. note that smaller models and the gpt-oss-120b style models they used to run are very prone to overthinking, so individual turns may be 3-10k tokens of just thinking + input + output. i don't think it's quite apples-to-apples to compare to a frontier model or even a k3. the odds of success (file compiles? read the right context?) are lower and thinking is longer.
- conception 1mo agoSo that’s about 400 tok/sec. Times that by 3, you get 100k in under a minute. That’s doing nothing special and just using your current setup.
- ricardobeat 1mo ago150k is 2500 tokens/sec.
- Aurornis 1mo ago> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. I don't understand. How does that make it unusable? Is the limit shared by an entire team at once? 150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find. I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.
- conception 1mo ago150k by account. At 1.5k a second you hit it very quickly.
- devy 1mo agoExactly, it burns the tokens 3000x faster, which means the budget ($$$$$$) runs out so faster it will stop super quick, not able to perform long-duration work. At 27B parameter size, the intelligence is not able to accomplish work within a short amount time. Consequently, it become not usable.
- gerdesj 1mo agoI (we) run Qwen3.8-27B-FP8 on a DGX Spark box - that's roughly £4000 of hardware. I did benchmark it in various ways and it runs quite well but it is a quantised jobbie and 1.5k t/s is also rather faster than anything I can possibly hope to achieve. To run that model at those sorts of speeds is going to need some serious investment and you are going to have to pay for it.
- jacquesm 1mo agoHow fast is it?
- 1mo ago
- puppymaster 1mo agoall the above. They just simply do not care about non enterprise customers. Today they announced qwen, guess what - it's also the same day they pulled Gemma off their shared tier. No migration notice and all developers are scrambling as we speak trying to migrate. They gave a soft head-ups on discord a week ago and when folks complained about zero-day migration they started saying 'you aren't suppose to build production app on shared tier'.
- ryukoposting 1mo agoOn discord? Jeez. How professional.
- vidarh 1mo ago[dead]
- lukewarm707 1mo agouse 3rd party marketplaces. cerebras is resold on vercel, openrouter and huggingface.