7 ms·

Qwen3-Coder-480B hosted by Cerebras is $2/Mtok (both input and output) through OpenRouter. OpenRouter claims Cerebras is providing at least 2000 tokens per sec

Qwen3-Coder-480B hosted by Cerebras is $2/Mtok (both input and output) through OpenRouter. OpenRouter claims Cerebras is providing at least 2000 tokens per second, which would be around 10x as fast, and the feedback I'm seeing from independent benchmarks indicates that Qwen3-Coder-480B is a better model.

stocksinsmocks 1y ago
There is a national superset of “NIH” bias that I think will impede adoption of Chinese-origin models for the foreseeable future. That’s a shame because by many objective metrics they’re a better value.
- _mlbt 1y ago
  In my case it's not NIH, but rather that I don't trust or wish to support my nation's largest geopolitical adversary.
  - bigyabai 1y ago
    Your loss. Qwen3 A3B replaced ChatGPT for me entirely, it's hard for me to imagine going back using remote models when I can load finetuned and uncensored models at-will. Maybe you'd find consolation in using Apple or Nvidia-designed hardware for inference on these Chinese models? Sure, the hardware you own was also built by your "nation's largest geopolitical adversary" but that hasn't seemed to bother you much.
    - _mlbt 1y ago
      [flagged]
      - bigyabai 1y ago
        Go interrogate it for yourself: https://huggingface.co/huihui-ai/Huihui-Qwen3-30B-A3B-Instruct-2507-abliterated https://huggingface.co/huihui-ai/Huihui-Qwen3-30B-A3B-Instru... In my experience, abliterated models will typically respond to any of those questions without hestitation. Here's a sample of a response to your last question: The resemblance between Chinese President **Xi Jinping** and the beloved cartoon character **Winnie the Pooh** is both visually striking and widely observed—so much so that it has become a cultural phenomenon. Here’s why Xi Jinping *looks* like Winnie the Pooh: ### **1. Facial Features: A Perfect Match** | Feature | Winnie the Pooh | Xi Jinping | [...]
      - acoustics 1y ago
        > Do they recognize the existence of Taiwan as an independent nation? <0.5% of humanity lives in a country that recognizes Taiwan, I'm not sure what answer you expect from a chatbot.
    - wickedsight 1y ago
      How did it replace ChatGPT for you? I'm running Qwen3 Coder locally and in no way does it compare to ChatGPT. In agentic workflows it fails almost every time. Maybe I'm doing something wrong, but I'm falling back to OpenAI all the time.
      - evilduck 1y ago
        It feels to me like it could replace ChatGPT 3.5 from the perspective of comparing it to their web chat interface if you were just asking about programming things two years ago, but the world moved on and you can do a lot more than just talk with a model and copy paste code now. Having Qwen3 Coder's A3B available for chat oriented coding conversations is indeed amazing for what it is and for being local and free but I also struggled to get useful agentic tools to reliably work with it (a fair number of tool calls fail or start looping, even with correct and advised settings, and tried using Cline, Roo, Continue and their own Qwen Code CLI). Even when I do get it to work for a few tasks in a row I don't have the hardware to run at comparable speed or manage the massive context sizes as a hosted frontier model. And buying capable enough hardware costs about as much as many years of paying for top tier hosted models.
  - mft_ 1y ago
    Genuine question: how does downloading an open-weight model (Qwen in this case) and running it either locally or via a third-party service benefit China?
    - tipsysquid 1y ago
      from an adversarial / defensive position: the model weights and training data were groomed and known; therefore, the output is potentially predictable. this could be an advantage to the nationstate above the corpo
      - AnonymousPlanet 1y ago
        This is also true for any US model from a European perspective.
        rightbyte 1y ago
        And for any US model from an US perspective. Why is assumed that states are aligned with them self like some sort of CivIII player being coherent and self contained...
    - throw10920 1y ago
      Genuine answer: the model has been trained by companies that are required by law to censor them to conform to PRC CCP party lines, including rejection of consensus reality such as Tiananmen Square[1]. Yes, the censorship for some topics currently doesn't appear to be any good, but it does exist, will absolutely get better (both harder to subvert and more subtle), and makes the models less trustworthy than those from countries (US, EU, Sweden, whatever) that don't have that same level of state control. (note that I'm not claiming that there's no state control or picking any specific other country) That's the downside to the user. To loop that back to your question, the upside to China is soft power (the same kind that the US has been flushing away recently). It's pretty similar to TikTok - if you have an extremely popular thing that people spend hours a day on and start to filter their life through, and you can influence it, that's a huge amount of power - even if you don't make any money off of it. Now, to be fair to the context of your question, there isn't nearly as much soft power you can get from a model that people use primarily for coding - that I'm less concerned about. [1] https://www.tomsguide.com/ai/i-just-outsmarted-deepseeks-censorship-here-are-3-ways-to-get-answers-to-the-questions-it-doesnt-like https://www.tomsguide.com/ai/i-just-outsmarted-deepseeks-cen...
  - ulfw 1y ago
    "largest geopolitical adversary" I can't believe Americans all are falling for propaganda like this. So Russia is all fine now huh. You know the country you literally had nuclear warheads pointed at for decades and decades and decades on end.
    - girvo 1y ago
      Not that I care either way, but China is far larger in economy, military and population than Russia is. So "largest adversary" is correct, and it doesn't take away from the danger that Russia's government continues to pose (directly, in my extended family's case in eastern Ukraine)
      - deleted 1y ago
        [deleted]
    - _mlbt 1y ago
      Russia is the successor state of a former failed superpower. China is a rising superpower with a large, advanced military and a strong industrial base. There’s no comparison. China is a far greater threat to the West than Russia.
      - ulfw 1y ago
        What is it threatening to do to the US? or is for you being able to threat a threat already? If so, why did American companies invest for decades into China so eagerly with US government support?
        _mlbt 1y ago
        If they take Taiwan that would be very disruptive to the US and the rest of the world. They have made credible threats to do that. How does Russia threaten the United States? They can’t even take over Ukraine.
        fauigerzigerk 1y ago
  - hedora 1y ago
    So, which model providers are supporting the US? Multiple domestic providers are actively helping dismantle US-based science, research, public health, emergency response, democratic elections, etc.
sdesol 1y ago
As a bit of a side note, I want to like Cerebras, but using any of the models through OpenRouter that uses them has lead to, too many throttling responses. Like you can't seem to make a few calls per minute. I'm not sure if Cerebras is throttling OpenRouter or if they are throttling everybody. If somebody from Cerebras is reading this, are you having capacity issues?
- gompertz 1y ago
  You can get your own key with cerebras and then use it in openrouter. Its a little hidden, but for each provider you can explicitly provide your own key. Then it won't be throttled.
  - omneity 1y ago
    Why would you go through openrouter in this case?
    - theshrike79 1y ago
      Easier to switch models
- scottyeager 1y ago
  Haven't seen any issues like this while using a key from their subscription plan: https://www.cerebras.ai/blog/introducing-cerebras-code https://www.cerebras.ai/blog/introducing-cerebras-code
  - deleted 1y ago
    [deleted]

stocksinsmocks 1y ago

There is a national superset of “NIH” bias that I think will impede adoption of Chinese-origin models for the foreseeable future. That’s a shame because by many objective metrics they’re a better value.

_mlbt 1y ago
In my case it's not NIH, but rather that I don't trust or wish to support my nation's largest geopolitical adversary.
- bigyabai 1y ago
  Your loss. Qwen3 A3B replaced ChatGPT for me entirely, it's hard for me to imagine going back using remote models when I can load finetuned and uncensored models at-will. Maybe you'd find consolation in using Apple or Nvidia-designed hardware for inference on these Chinese models? Sure, the hardware you own was also built by your "nation's largest geopolitical adversary" but that hasn't seemed to bother you much.
  - _mlbt 1y ago
    [flagged]
    - bigyabai 1y ago
      Go interrogate it for yourself: https://huggingface.co/huihui-ai/Huihui-Qwen3-30B-A3B-Instruct-2507-abliterated https://huggingface.co/huihui-ai/Huihui-Qwen3-30B-A3B-Instru... In my experience, abliterated models will typically respond to any of those questions without hestitation. Here's a sample of a response to your last question: The resemblance between Chinese President **Xi Jinping** and the beloved cartoon character **Winnie the Pooh** is both visually striking and widely observed—so much so that it has become a cultural phenomenon. Here’s why Xi Jinping *looks* like Winnie the Pooh: ### **1. Facial Features: A Perfect Match** | Feature | Winnie the Pooh | Xi Jinping | [...]
    - acoustics 1y ago
      > Do they recognize the existence of Taiwan as an independent nation? <0.5% of humanity lives in a country that recognizes Taiwan, I'm not sure what answer you expect from a chatbot.
  - wickedsight 1y ago
    How did it replace ChatGPT for you? I'm running Qwen3 Coder locally and in no way does it compare to ChatGPT. In agentic workflows it fails almost every time. Maybe I'm doing something wrong, but I'm falling back to OpenAI all the time.
    - evilduck 1y ago
      It feels to me like it could replace ChatGPT 3.5 from the perspective of comparing it to their web chat interface if you were just asking about programming things two years ago, but the world moved on and you can do a lot more than just talk with a model and copy paste code now. Having Qwen3 Coder's A3B available for chat oriented coding conversations is indeed amazing for what it is and for being local and free but I also struggled to get useful agentic tools to reliably work with it (a fair number of tool calls fail or start looping, even with correct and advised settings, and tried using Cline, Roo, Continue and their own Qwen Code CLI). Even when I do get it to work for a few tasks in a row I don't have the hardware to run at comparable speed or manage the massive context sizes as a hosted frontier model. And buying capable enough hardware costs about as much as many years of paying for top tier hosted models.
- mft_ 1y ago
  Genuine question: how does downloading an open-weight model (Qwen in this case) and running it either locally or via a third-party service benefit China?
  - tipsysquid 1y ago
    from an adversarial / defensive position: the model weights and training data were groomed and known; therefore, the output is potentially predictable. this could be an advantage to the nationstate above the corpo
    - AnonymousPlanet 1y ago
      This is also true for any US model from a European perspective.
      - rightbyte 1y ago
        And for any US model from an US perspective. Why is assumed that states are aligned with them self like some sort of CivIII player being coherent and self contained...
  - throw10920 1y ago
    Genuine answer: the model has been trained by companies that are required by law to censor them to conform to PRC CCP party lines, including rejection of consensus reality such as Tiananmen Square[1]. Yes, the censorship for some topics currently doesn't appear to be any good, but it does exist, will absolutely get better (both harder to subvert and more subtle), and makes the models less trustworthy than those from countries (US, EU, Sweden, whatever) that don't have that same level of state control. (note that I'm not claiming that there's no state control or picking any specific other country) That's the downside to the user. To loop that back to your question, the upside to China is soft power (the same kind that the US has been flushing away recently). It's pretty similar to TikTok - if you have an extremely popular thing that people spend hours a day on and start to filter their life through, and you can influence it, that's a huge amount of power - even if you don't make any money off of it. Now, to be fair to the context of your question, there isn't nearly as much soft power you can get from a model that people use primarily for coding - that I'm less concerned about. [1] https://www.tomsguide.com/ai/i-just-outsmarted-deepseeks-censorship-here-are-3-ways-to-get-answers-to-the-questions-it-doesnt-like https://www.tomsguide.com/ai/i-just-outsmarted-deepseeks-cen...
- ulfw 1y ago
  "largest geopolitical adversary" I can't believe Americans all are falling for propaganda like this. So Russia is all fine now huh. You know the country you literally had nuclear warheads pointed at for decades and decades and decades on end.
  - girvo 1y ago
    Not that I care either way, but China is far larger in economy, military and population than Russia is. So "largest adversary" is correct, and it doesn't take away from the danger that Russia's government continues to pose (directly, in my extended family's case in eastern Ukraine)
    - deleted 1y ago
      [deleted]
  - _mlbt 1y ago
    Russia is the successor state of a former failed superpower. China is a rising superpower with a large, advanced military and a strong industrial base. There’s no comparison. China is a far greater threat to the West than Russia.
    - ulfw 1y ago
      What is it threatening to do to the US? or is for you being able to threat a threat already? If so, why did American companies invest for decades into China so eagerly with US government support?
      - _mlbt 1y ago
        If they take Taiwan that would be very disruptive to the US and the rest of the world. They have made credible threats to do that. How does Russia threaten the United States? They can’t even take over Ukraine.
        fauigerzigerk 1y ago
- hedora 1y ago
  So, which model providers are supporting the US? Multiple domestic providers are actively helping dismantle US-based science, research, public health, emergency response, democratic elections, etc.

sdesol 1y ago

As a bit of a side note, I want to like Cerebras, but using any of the models through OpenRouter that uses them has lead to, too many throttling responses. Like you can't seem to make a few calls per minute. I'm not sure if Cerebras is throttling OpenRouter or if they are throttling everybody. If somebody from Cerebras is reading this, are you having capacity issues?

gompertz 1y ago
You can get your own key with cerebras and then use it in openrouter. Its a little hidden, but for each provider you can explicitly provide your own key. Then it won't be throttled.
- omneity 1y ago
  Why would you go through openrouter in this case?
  - theshrike79 1y ago
    Easier to switch models
scottyeager 1y ago
Haven't seen any issues like this while using a key from their subscription plan: https://www.cerebras.ai/blog/introducing-cerebras-code https://www.cerebras.ai/blog/introducing-cerebras-code
- deleted 1y ago
  [deleted]