4 ms·

I love the Chinese models. I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks. DeepSeek v4 Flash is e

I love the Chinese models. I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks. DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform). I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up. It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage. OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.

copperx 2mo ago
What don't you like about the service, besides the mark up?
- apatheticonion 2mo ago
  For me, the only utility OpenRouter gives me is billing consolidation - I don't really need the routing capabilities because I use Bifrost for that. As a router, it's not very feature rich. For example I restricted the available models to the ones I want to use however the `/models` endpoint still lists all the models, making my LLM client list the 200+ models available on the service (even though they will throw an error if I try to use them). With Bifrost, I can also create model aliases with custom configuration - for example I can create a model alias `deepseek-v4-flash-nothink` which disables thinking. I can create `deepseek-v4-flash-caveman` which injects the caveman skill (to save tokens) etc. Plus I can contribute to Bifrost, which I can't do with OpenRouter.
  - robbiet480 2mo ago
    Did you look at LiteLLM at all? It seems fine but Bifrost looks interesting too.
    - apatheticonion 2mo ago
      Yeah I looked at it, LiteLLM is functionally more mature. The only reason I didn't go for it is I'm not a fan of Python dependency management, Bifrost is just a single executable that uses nearly no memory and is lightning fast.
    - CuriouslyC 2mo ago
      LiteLLM is bad. Shit performance, buggy, and none of it surprising if you look at the tangled mess that is their codebase.
  - aftbit 2mo ago
    Can you configure Bifrost using entirely config files without the web interface? Can it run without a database or anything stateful? I'm using LiteLLM but it is not trivial to run.
    - apatheticonion 2mo ago
      You can configure it with just config files. Database is optional, if you add one, you get advanced caching. It's just a single executable. wget https://downloads.getmaxim.ai/bifrost/v1.6.5/linux/amd64/bifrost-http chmod +x bifrost-http ./bifrost-http
- schneehertz 2mo ago
  openrouter’s API has removed parameters specific to certain models, impairing model functionality. For example, the image generation parameters of GPT-image-2 are largely ineffective.
ronsor 2mo ago
Have you looked into OpenCode Zen or OpenCode Go? https://opencode.ai/zen https://opencode.ai/zen
- samtp 2mo ago
  there is also https://cline.bot/cline-pass https://cline.bot/cline-pass
  - verdverm 2mo ago
    basically the same deal as OC Go, lending credence to token commodification
- criley2 2mo ago
  Zen is nice, but they require US hosting so they don't get new Chinese models right away. There is no Kimi K3. Go is nice for the ten minutes you can use it until your hit your cap.
  - kzrdude 2mo ago
    Go is great if you prefer deepseek V4 flash. Then it's very hard to use up the allowance.
arikrahman 2mo ago
The best part is using harnesses like reasonix or whale make cache hit at a rate close to 98%, making requests converge to practically free. And that's with unsubsidized American providers like cloudflare or Digital Ocean.
- mlazos 2mo ago
  How can you be hitting cache on what I think are novel LLM prompts …
  - oceansweep 2mo ago
    Not them but my understanding is that the harness will send a simple 'heartbeat' message to keep the cache 'warm', (see prefix caching: https://handbook.modular.com/inference-optimization/prefix-caching/ https://handbook.modular.com/inference-optimization/prefix-c... ) which can then be edited/changed, which does cause the user to incur a fee, but its much less than the amount they'd pay on a no-cache hit request.
  - verdverm 2mo ago
    multi turn sessions, they are typically in the high 90% hit rate across all providers without doing much of anything
  - slopinthebag 2mo ago
    The prompt is only novel the first time it’s sent. Then as it’s sent repeatedly as part of the previous context it’s cached. Deepseek afaik has a novel architecture that is somewhat forgiving of cache shifts. I’ve been getting +90% cache hits with Zed’s agent and it’s not doing anything special regarding caching afaik.
- sghiassy 2mo ago
  I’ve never used either of those tools. Do they work well?
- KronisLV 2mo ago
  In long form tasks, across multiple harnesses (Claude Code, OpenCode, Kimi Code, ZCode) my cache rates are typically 96-99%. I don’t think that’s particularly out of the ordinary. Do people have different experiences with other harnesses? Which ones?
skeledrew 2mo ago
Prepaid means someone can't sign up, use a bunch of inference, then cancel their card and disappear into the sunset. VPS is a more long term investment where it's harder to switch and it doesn't cost the provider much if a few users jump out without paying for a month.
- apatheticonion 2mo ago
  I do understand the reasoning for it, it still sucks as a user who is doing the right thing.
bellowsgulch 2mo ago
[flagged]
- apatheticonion 2mo ago
  It's more just the value for money. It doesn't really offer me anything valuable other than billing consolidation - and the markup is excessive for that use case. If I was using the routing features it would make more sense - but they charge significantly more for that so the markup is really just for billing consolidation.
  - deleted 2mo ago
    [deleted]
- dang 2mo ago
  "Don't be snarky." "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Der_Einzige 2mo ago
  "After all, anyone can build an OpenAI API compatible server that works with every major provider, and even build it using an LLM at that!" This but UNIRONICALLY lmao!!!
  - deleted 2mo ago
    [deleted]
dools 2mo ago
Yeah deepseek is my workhorse of choice these days, I mostly use pro because I do a lot of concurrent jobs so speed is less of a concern. When it falls over I change the model mid-chat to GPT5.5 and chuck a couple of tokens over to OpenAI and then once it's correctly found the problem I switch back to deepseek, keeping all the 5.5 analysis in context. I have been a heavy user of k2.5/6 but they seem to have gotten slower and worse at their jobs (and the "engine overloaded" errors have increased ... ), and k3 spends a LOT of time thinking and doesn't produce noticeably better results. I do keep chucking some tasks over to moonshot every now and again to give it a chance. I figure they're under compute pressure and have probably quantised the models to cope temporarily. But recently I've found myself mostly using deepseek v4 pro and gpt 5.5 when I need the big guns. GLM 5.2 has been good at select tasks but really shit at others and given how sparingly I use GPT 5.5 I'm not that compelled to use it. I've yet to be impressed by any Google or Anthropic models!!
- MrBuddyCasino 2mo ago
  > GLM 5.2 has been good at select tasks but really shit at others Yeah it really likes to shit the bed after thinking forever. Even if it can do some things, the quality is not quite up there in my experience.
adamisnotroman 2mo ago
I didn't realize that OpenRouter had a mark up. Is it a flat mark up across the board or depending on the model?
- apatheticonion 2mo ago
  Under $20 it's a flat $0.8 fee. Over $20 it's +5.5% + credit card charges (+1.5%)
equinumerous 2mo ago
As someone who is also eyeing Bifrost, I am curious to know what made you choose Bifrost at all / why you decided to use an LLM gateway. I also was considering OpenRouter, since it provides pretty much every model, with same-day releases for new models. One thing that is attractive to me about Bifrost is that if I decide to leave OpenRouter tomorrow, I can do so without touching any other part of my stack; I'm hoping self-hosting models becomes more viable, and then I can become less dependent on third party LLM providers like OpenAI/Anthropic/OpenRouter, and I would not need to worry about a model that I depend on suddenly being deprecated.

copperx 2mo ago

What don't you like about the service, besides the mark up?

apatheticonion 2mo ago
For me, the only utility OpenRouter gives me is billing consolidation - I don't really need the routing capabilities because I use Bifrost for that. As a router, it's not very feature rich. For example I restricted the available models to the ones I want to use however the `/models` endpoint still lists all the models, making my LLM client list the 200+ models available on the service (even though they will throw an error if I try to use them). With Bifrost, I can also create model aliases with custom configuration - for example I can create a model alias `deepseek-v4-flash-nothink` which disables thinking. I can create `deepseek-v4-flash-caveman` which injects the caveman skill (to save tokens) etc. Plus I can contribute to Bifrost, which I can't do with OpenRouter.
- robbiet480 2mo ago
  Did you look at LiteLLM at all? It seems fine but Bifrost looks interesting too.
  - apatheticonion 2mo ago
    Yeah I looked at it, LiteLLM is functionally more mature. The only reason I didn't go for it is I'm not a fan of Python dependency management, Bifrost is just a single executable that uses nearly no memory and is lightning fast.
  - CuriouslyC 2mo ago
    LiteLLM is bad. Shit performance, buggy, and none of it surprising if you look at the tangled mess that is their codebase.
- aftbit 2mo ago
  Can you configure Bifrost using entirely config files without the web interface? Can it run without a database or anything stateful? I'm using LiteLLM but it is not trivial to run.
  - apatheticonion 2mo ago
    You can configure it with just config files. Database is optional, if you add one, you get advanced caching. It's just a single executable. wget https://downloads.getmaxim.ai/bifrost/v1.6.5/linux/amd64/bifrost-http chmod +x bifrost-http ./bifrost-http
schneehertz 2mo ago
openrouter’s API has removed parameters specific to certain models, impairing model functionality. For example, the image generation parameters of GPT-image-2 are largely ineffective.

ronsor 2mo ago

Have you looked into OpenCode Zen or OpenCode Go? https://opencode.ai/zen https://opencode.ai/zen

samtp 2mo ago
there is also https://cline.bot/cline-pass https://cline.bot/cline-pass
- verdverm 2mo ago
  basically the same deal as OC Go, lending credence to token commodification
criley2 2mo ago
Zen is nice, but they require US hosting so they don't get new Chinese models right away. There is no Kimi K3. Go is nice for the ten minutes you can use it until your hit your cap.
- kzrdude 2mo ago
  Go is great if you prefer deepseek V4 flash. Then it's very hard to use up the allowance.

arikrahman 2mo ago

The best part is using harnesses like reasonix or whale make cache hit at a rate close to 98%, making requests converge to practically free. And that's with unsubsidized American providers like cloudflare or Digital Ocean.

mlazos 2mo ago
How can you be hitting cache on what I think are novel LLM prompts …
- oceansweep 2mo ago
  Not them but my understanding is that the harness will send a simple 'heartbeat' message to keep the cache 'warm', (see prefix caching: https://handbook.modular.com/inference-optimization/prefix-caching/ https://handbook.modular.com/inference-optimization/prefix-c... ) which can then be edited/changed, which does cause the user to incur a fee, but its much less than the amount they'd pay on a no-cache hit request.
- verdverm 2mo ago
  multi turn sessions, they are typically in the high 90% hit rate across all providers without doing much of anything
- slopinthebag 2mo ago
  The prompt is only novel the first time it’s sent. Then as it’s sent repeatedly as part of the previous context it’s cached. Deepseek afaik has a novel architecture that is somewhat forgiving of cache shifts. I’ve been getting +90% cache hits with Zed’s agent and it’s not doing anything special regarding caching afaik.
sghiassy 2mo ago
I’ve never used either of those tools. Do they work well?
KronisLV 2mo ago
In long form tasks, across multiple harnesses (Claude Code, OpenCode, Kimi Code, ZCode) my cache rates are typically 96-99%. I don’t think that’s particularly out of the ordinary. Do people have different experiences with other harnesses? Which ones?

skeledrew 2mo ago

Prepaid means someone can't sign up, use a bunch of inference, then cancel their card and disappear into the sunset. VPS is a more long term investment where it's harder to switch and it doesn't cost the provider much if a few users jump out without paying for a month.

apatheticonion 2mo ago
I do understand the reasoning for it, it still sucks as a user who is doing the right thing.

bellowsgulch 2mo ago

[flagged]

apatheticonion 2mo ago
It's more just the value for money. It doesn't really offer me anything valuable other than billing consolidation - and the markup is excessive for that use case. If I was using the routing features it would make more sense - but they charge significantly more for that so the markup is really just for billing consolidation.
- deleted 2mo ago
  [deleted]
dang 2mo ago
"Don't be snarky." "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
Der_Einzige 2mo ago
"After all, anyone can build an OpenAI API compatible server that works with every major provider, and even build it using an LLM at that!" This but UNIRONICALLY lmao!!!
- deleted 2mo ago
  [deleted]

dools 2mo ago

Yeah deepseek is my workhorse of choice these days, I mostly use pro because I do a lot of concurrent jobs so speed is less of a concern. When it falls over I change the model mid-chat to GPT5.5 and chuck a couple of tokens over to OpenAI and then once it's correctly found the problem I switch back to deepseek, keeping all the 5.5 analysis in context. I have been a heavy user of k2.5/6 but they seem to have gotten slower and worse at their jobs (and the "engine overloaded" errors have increased ... ), and k3 spends a LOT of time thinking and doesn't produce noticeably better results. I do keep chucking some tasks over to moonshot every now and again to give it a chance. I figure they're under compute pressure and have probably quantised the models to cope temporarily. But recently I've found myself mostly using deepseek v4 pro and gpt 5.5 when I need the big guns. GLM 5.2 has been good at select tasks but really shit at others and given how sparingly I use GPT 5.5 I'm not that compelled to use it. I've yet to be impressed by any Google or Anthropic models!!

MrBuddyCasino 2mo ago
> GLM 5.2 has been good at select tasks but really shit at others Yeah it really likes to shit the bed after thinking forever. Even if it can do some things, the quality is not quite up there in my experience.

adamisnotroman 2mo ago

I didn't realize that OpenRouter had a mark up. Is it a flat mark up across the board or depending on the model?

apatheticonion 2mo ago
Under $20 it's a flat $0.8 fee. Over $20 it's +5.5% + credit card charges (+1.5%)

equinumerous 2mo ago

As someone who is also eyeing Bifrost, I am curious to know what made you choose Bifrost at all / why you decided to use an LLM gateway. I also was considering OpenRouter, since it provides pretty much every model, with same-day releases for new models. One thing that is attractive to me about Bifrost is that if I decide to leave OpenRouter tomorrow, I can do so without touching any other part of my stack; I'm hoping self-hosting models becomes more viable, and then I can become less dependent on third party LLM providers like OpenAI/Anthropic/OpenRouter, and I would not need to worry about a model that I depend on suddenly being deprecated.