5 ms·

> The best "free" experience I've found is using OpenCode with Big Pickle. I have absolutely zero interest in free. I honestly don't think I'm even remotely in

by rapind 4mo ago

> The best "free" experience I've found is using OpenCode with Big Pickle. I have absolutely zero interest in free. I honestly don't think I'm even remotely in the same demographic as people using free tiers / models. I want to pay. I don't want my data used for training. I want it to be open. I want it to be consistently up (more than Claude!). I want it to be fast. I don't want it to be subsidized as that's just an excuse for shitty quality. Deepseek flash knocks it out of the park on all of these except you're data is used in training. I'm fine with it being hosted since there's no way I'm using it 24/7, but data MUST be private. Basically I want Hetzner and OVH to run open model clouds. I'm convinced this is going to happen eventually when everyone realizes this is a commodity.

Bnjoroge 4mo ago
You can specify which providers you want to serve your model in OpenRouter. Then you can chose US-based ones.
aamoscodes 4mo ago
You can pay, and also use deepseek-v4-flash. OpenRouter even lets you "block" or limit your usage to providers that don't train on data. Since the weights are open, other companies are already serving the model on non-DeepSeek owned hardware: https://openrouter.ai/deepseek/deepseek-v4-flash https://openrouter.ai/deepseek/deepseek-v4-flash
- rapind 4mo ago
  Good to know. I hadn't checks since early is DS4's launch when they were the only provide (I think maybe there was one other, but they also trained on your data). I see several private options now.
- fc417fc802 4mo ago
  > OpenRouter even lets you "block" or limit your usage to providers that don't train on data. More than that, they have various zero data retention options and provide a convenient json list of them.
  - larodi 4mo ago
    The fact OpenRouter strips https to reroute screams danger already.
    - fc417fc802 4mo ago
      What do you mean? Are you objecting that they communicate with the provider on your behalf? But how else would you design such a system? Plumbing you straight through would require nonstandard certificate juggling and they wouldn't be able to implement their core service of providing a standardized API nor could they transparently route your request to the fastest / cheapest / whatever provider on the fly nor could they implement transparent fallback nor could they implement their policy of not billing you if the response from the provider is invalid. Also the chosen provider could fingerprint your network stack if you communicated directly. The routing service is acting as a proxy and for most providers fully anonymizes requests (it does send a stable uid to some of them though).
      - larodi 4mo ago
        Precisely this. That somehow is okay to put your trust into man in the middle. To your comment how you do it - yes it is difficult to do it right, but not impossible.
darkmarmot 4mo ago
Hard to guarantee it's private if you don't keep it local... I don't have a lot of trust for companies in this space.
- rapind 4mo ago
  Yes, but I think that'll change eventually. If you trust hosting your code with a specific cloud provider then you'll probably also trust them for code assist. At least that's my theory. There'll probably need to be a threat of massive litigation should they fail to comply with such a policy.
  - pessimizer 4mo ago
    > If you trust hosting your code with a specific cloud provider then you'll probably also trust them for code assist. I'm interested in this thought. There is significant motivation for providers to create a verifiable way for them not to deal with having access to client interactions with LLMs at all. Whatever standards and protocols have to be come up with in order to reassure clients. Any good standards for privacy when interacting with LLMs could also trickle down to smaller providers, and everyone could offer guarantees. Even if the guarantee was literally just an insurance policy and a private court to decide if it pays out.
  - naikrovek 4mo ago
    > Yes, but I think that'll change eventually. Maybe people will trust companies, but those companies will rarely deserve that trust. Anyone that pays attention sees breach announcements almost every day. Security is never a concern for these companies until it embarrasses them. Then, as soon as the negative attention fades, security again becomes the second to last priority. Do not trust companies with any data that is important to you unless the effective management of that data is required by law, and the laws are comprehensive.
    - fc417fc802 4mo ago
      If your contract says there's no data retention and then a bunch of your retained data gets leaked in a breach presumably you have grounds for a lawsuit.
bel8 4mo ago
These competent open models you want to use were trained on data from people like you and me. I wonder if there are competent models trained purely on permissive open-source code like MIT or Apache 2.0.
- yencabulator 4mo ago
  MIT and Apache 2.0 both require attribution, so it's not like limiting to those would help in license compliance.
saghm 4mo ago
I'm probably somewhat adjacent to you. I would be happy to pay, but I just don't want to pay any of the companies that are actually offering things right now. I had the $20/month sub for Claude for a couple months, until one day I kept inexplicably getting errors saying I hit the limit even though their site showed my usage at less than half for the session and 8% for the week, and it seemed silly to pay for something that couldn't even properly respect its own measurements. OpenAI sketches me out too much as a company, Cursor feels lackluster when I use it for work from the account they pay for (and now is getting acquired by maybe the only AI company even sketchier than OpenAI), and I wasn't particularly impressed with Gemini or Mistral Vibe either when I tried them on the free tiers either.
- rapind 4mo ago
  I was paying around $500 / month on average between multiple providers for over a year. I cancelled one a while ago because of pretty bad service availability (Bet you guess who that is!), which by all reports hasn't improved much. For me, paying from $200 - $500 / month is reasonable if I can sustain a disruption free flow that doesn't require constant yak shaving. What I've found experimenting with DeepSeek on some open source library stuff is that it's actually going to cost me much less if I don't need frontier vibing (which I don't).
  - gaolei8888 4mo ago
    who?
deleted 4mo ago
[deleted]
rlkf 4mo ago
> Basically I want Hetzner and OVH to run open model clouds You can run Qwen3 on OVH already: <https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/ https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalo...>
- johndough 4mo ago
  I see that OVH offers Qwen3.5-397B-A17B, which is a bit surprising to me. I thought that EU providers had to comply with the AI act where you have to provide opt-out and information about the training data once the model is sufficiently large (over 10^23 FLOPs, likely the case here), but providing information is not possible since people who train those models only give vague information at best. Does anyone know if OVH is ignoring the law here, or whether it does not apply for some reason?
  - dofm 4mo ago
    Which law is that? Not doubting you — just want to read it!
    - johndough 4mo ago
      Article 53 of the AI Act: https://ai-act-law.eu/article/53/ https://ai-act-law.eu/article/53/ The definition of a "genral-purpose AI model" is described in more detail in the "Guidelines on the scope of obligations for providers of general-purpose AI models under the AI Act": https://ec.europa.eu/newsroom/dae/redirection/document/118340 https://ec.europa.eu/newsroom/dae/redirection/document/11834...
      - dofm 4mo ago
        Thanks, v. interesting.
      - zwaps 4mo ago
        Does not apply to oss models
milesvp 4mo ago
If you think your data isn’t being hoovered up I’d like to point out that every model is possible due to federal crimes committed to obtain the information they were trained on. Regardless of how much you are paying, your data is worth another petty civil infraction.
- horacemorace 4mo ago
  A million times this. There is “private” as a corporate-legality licensing perspective. There is “private” as a human concept. The two are seemingly opposite, yet as all the money is focused on the former there’s no airtime left for the latter.
  - suncemoje 4mo ago
    Then I'm interested if there are any facts as to what ZDR actually means?
    - jaapz 4mo ago
      It can still mean Zero Data Retention - i just comes down to whether you trust the company to actually do what they promise. The fact that they've trained models on data that wasn't theirs does not make me trust them a lot when they make this claim.
      - munksbeer 4mo ago
        When discussing this, may I ask (I know you are probably bored of the actual arguments), what does "trained models on data that wasn't theirs" actually mean in practice? Again, I know these arguments have been done to death, but every human who reads source code that wasn't written by them, or views art that wasn't created by them, and practices against this art, is training their brain on data "that wasn't theirs". They are frequently making a living doing so. Is this distinction the scale, or is there actually a different more strict definition that we should be using as a common language to talk about this? As in, I should not even be reading certain source code if it is not licensed appropriate, or I will be in breach because I'm training myself illegally? And the same question for art, etc?
djmips 4mo ago
Did you try Claude Fable?
superze 4mo ago
Hetzner workforce can barely run a mature technology called s3 and you think they will be able to deploy openmodels?
- KronisLV 4mo ago
  What mature implementations of S3 are there? MinIO that rugpulled the community, Garage that doesn’t even have proper setup scripts in their Docker containers and expect you to do the init manually, or Zenko cloud server that more or less got abandoned? I think there’s also SeaweedFS which might do better but I’m surprised at how shitty everything seems in this space - surely people aren’t being crazy and either storing their files on the FS directly to expose access to them through their app (hello directory traversal attacks) or storing them in relational DBs (hello wasted bandwidth and bloated backups). The odd jank extends further, like Sonatype Nexus and some other software hardcodes AWS regions to choose from when configuring the storage even though your self-hosted implementation doesn’t have anything to do with AWS so you just have to come up with fake regions. If the cloud vendors each have to reimplement it because there is nothing as quality as PostgreSQL is for DBs, but for S3, then I’m hardly surprised at the state of things.
  - chrislusf 4mo ago
    I work on SeaweedFS. Let me know if see any bugs or just create a github issue.
gb2d_hn 4mo ago
For me it's about the value of my time. I think that it's important that we have open models, but for getting real work done, my time is too valuable to waste it on subpar results or additional agent management when a max plan covers all the use I need. It's not worth quibbling over. If the cost / benefit ratio changes, I'll be looking harder at local set ups, but not at the moment.
dvngnt_ 4mo ago
Would venace ai work?

by rapind 4mo ago

Bnjoroge 4mo ago

You can specify which providers you want to serve your model in OpenRouter. Then you can chose US-based ones.

aamoscodes 4mo ago

You can pay, and also use deepseek-v4-flash. OpenRouter even lets you "block" or limit your usage to providers that don't train on data. Since the weights are open, other companies are already serving the model on non-DeepSeek owned hardware: https://openrouter.ai/deepseek/deepseek-v4-flash https://openrouter.ai/deepseek/deepseek-v4-flash

rapind 4mo ago
Good to know. I hadn't checks since early is DS4's launch when they were the only provide (I think maybe there was one other, but they also trained on your data). I see several private options now.
fc417fc802 4mo ago
> OpenRouter even lets you "block" or limit your usage to providers that don't train on data. More than that, they have various zero data retention options and provide a convenient json list of them.
- larodi 4mo ago
  The fact OpenRouter strips https to reroute screams danger already.
  - fc417fc802 4mo ago
    What do you mean? Are you objecting that they communicate with the provider on your behalf? But how else would you design such a system? Plumbing you straight through would require nonstandard certificate juggling and they wouldn't be able to implement their core service of providing a standardized API nor could they transparently route your request to the fastest / cheapest / whatever provider on the fly nor could they implement transparent fallback nor could they implement their policy of not billing you if the response from the provider is invalid. Also the chosen provider could fingerprint your network stack if you communicated directly. The routing service is acting as a proxy and for most providers fully anonymizes requests (it does send a stable uid to some of them though).
    - larodi 4mo ago
      Precisely this. That somehow is okay to put your trust into man in the middle. To your comment how you do it - yes it is difficult to do it right, but not impossible.

darkmarmot 4mo ago

Hard to guarantee it's private if you don't keep it local... I don't have a lot of trust for companies in this space.

rapind 4mo ago
Yes, but I think that'll change eventually. If you trust hosting your code with a specific cloud provider then you'll probably also trust them for code assist. At least that's my theory. There'll probably need to be a threat of massive litigation should they fail to comply with such a policy.
- pessimizer 4mo ago
  > If you trust hosting your code with a specific cloud provider then you'll probably also trust them for code assist. I'm interested in this thought. There is significant motivation for providers to create a verifiable way for them not to deal with having access to client interactions with LLMs at all. Whatever standards and protocols have to be come up with in order to reassure clients. Any good standards for privacy when interacting with LLMs could also trickle down to smaller providers, and everyone could offer guarantees. Even if the guarantee was literally just an insurance policy and a private court to decide if it pays out.
- naikrovek 4mo ago
  > Yes, but I think that'll change eventually. Maybe people will trust companies, but those companies will rarely deserve that trust. Anyone that pays attention sees breach announcements almost every day. Security is never a concern for these companies until it embarrasses them. Then, as soon as the negative attention fades, security again becomes the second to last priority. Do not trust companies with any data that is important to you unless the effective management of that data is required by law, and the laws are comprehensive.
  - fc417fc802 4mo ago
    If your contract says there's no data retention and then a bunch of your retained data gets leaked in a breach presumably you have grounds for a lawsuit.

bel8 4mo ago

These competent open models you want to use were trained on data from people like you and me. I wonder if there are competent models trained purely on permissive open-source code like MIT or Apache 2.0.

yencabulator 4mo ago
MIT and Apache 2.0 both require attribution, so it's not like limiting to those would help in license compliance.

saghm 4mo ago

I'm probably somewhat adjacent to you. I would be happy to pay, but I just don't want to pay any of the companies that are actually offering things right now. I had the $20/month sub for Claude for a couple months, until one day I kept inexplicably getting errors saying I hit the limit even though their site showed my usage at less than half for the session and 8% for the week, and it seemed silly to pay for something that couldn't even properly respect its own measurements. OpenAI sketches me out too much as a company, Cursor feels lackluster when I use it for work from the account they pay for (and now is getting acquired by maybe the only AI company even sketchier than OpenAI), and I wasn't particularly impressed with Gemini or Mistral Vibe either when I tried them on the free tiers either.

rapind 4mo ago
I was paying around $500 / month on average between multiple providers for over a year. I cancelled one a while ago because of pretty bad service availability (Bet you guess who that is!), which by all reports hasn't improved much. For me, paying from $200 - $500 / month is reasonable if I can sustain a disruption free flow that doesn't require constant yak shaving. What I've found experimenting with DeepSeek on some open source library stuff is that it's actually going to cost me much less if I don't need frontier vibing (which I don't).
- gaolei8888 4mo ago
  who?

deleted 4mo ago

[deleted]

rlkf 4mo ago

> Basically I want Hetzner and OVH to run open model clouds You can run Qwen3 on OVH already: <https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/ https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalo...>

johndough 4mo ago
I see that OVH offers Qwen3.5-397B-A17B, which is a bit surprising to me. I thought that EU providers had to comply with the AI act where you have to provide opt-out and information about the training data once the model is sufficiently large (over 10^23 FLOPs, likely the case here), but providing information is not possible since people who train those models only give vague information at best. Does anyone know if OVH is ignoring the law here, or whether it does not apply for some reason?
- dofm 4mo ago
  Which law is that? Not doubting you — just want to read it!
  - johndough 4mo ago
    Article 53 of the AI Act: https://ai-act-law.eu/article/53/ https://ai-act-law.eu/article/53/ The definition of a "genral-purpose AI model" is described in more detail in the "Guidelines on the scope of obligations for providers of general-purpose AI models under the AI Act": https://ec.europa.eu/newsroom/dae/redirection/document/118340 https://ec.europa.eu/newsroom/dae/redirection/document/11834...
    - dofm 4mo ago
      Thanks, v. interesting.
    - zwaps 4mo ago
      Does not apply to oss models

milesvp 4mo ago

If you think your data isn’t being hoovered up I’d like to point out that every model is possible due to federal crimes committed to obtain the information they were trained on. Regardless of how much you are paying, your data is worth another petty civil infraction.

horacemorace 4mo ago
A million times this. There is “private” as a corporate-legality licensing perspective. There is “private” as a human concept. The two are seemingly opposite, yet as all the money is focused on the former there’s no airtime left for the latter.
- suncemoje 4mo ago
  Then I'm interested if there are any facts as to what ZDR actually means?
  - jaapz 4mo ago
    It can still mean Zero Data Retention - i just comes down to whether you trust the company to actually do what they promise. The fact that they've trained models on data that wasn't theirs does not make me trust them a lot when they make this claim.
    - munksbeer 4mo ago
      When discussing this, may I ask (I know you are probably bored of the actual arguments), what does "trained models on data that wasn't theirs" actually mean in practice? Again, I know these arguments have been done to death, but every human who reads source code that wasn't written by them, or views art that wasn't created by them, and practices against this art, is training their brain on data "that wasn't theirs". They are frequently making a living doing so. Is this distinction the scale, or is there actually a different more strict definition that we should be using as a common language to talk about this? As in, I should not even be reading certain source code if it is not licensed appropriate, or I will be in breach because I'm training myself illegally? And the same question for art, etc?

djmips 4mo ago

Did you try Claude Fable?

superze 4mo ago

Hetzner workforce can barely run a mature technology called s3 and you think they will be able to deploy openmodels?

KronisLV 4mo ago
What mature implementations of S3 are there? MinIO that rugpulled the community, Garage that doesn’t even have proper setup scripts in their Docker containers and expect you to do the init manually, or Zenko cloud server that more or less got abandoned? I think there’s also SeaweedFS which might do better but I’m surprised at how shitty everything seems in this space - surely people aren’t being crazy and either storing their files on the FS directly to expose access to them through their app (hello directory traversal attacks) or storing them in relational DBs (hello wasted bandwidth and bloated backups). The odd jank extends further, like Sonatype Nexus and some other software hardcodes AWS regions to choose from when configuring the storage even though your self-hosted implementation doesn’t have anything to do with AWS so you just have to come up with fake regions. If the cloud vendors each have to reimplement it because there is nothing as quality as PostgreSQL is for DBs, but for S3, then I’m hardly surprised at the state of things.
- chrislusf 4mo ago
  I work on SeaweedFS. Let me know if see any bugs or just create a github issue.

gb2d_hn 4mo ago

For me it's about the value of my time. I think that it's important that we have open models, but for getting real work done, my time is too valuable to waste it on subpar results or additional agent management when a max plan covers all the use I need. It's not worth quibbling over. If the cost / benefit ratio changes, I'll be looking harder at local set ups, but not at the moment.

dvngnt_ 4mo ago

Would venace ai work?