3 ms·

You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you

You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider?

txrx0000 18d ago
Technically true, but the delay between local and closed frontier is only a few months. And individual sovereignty / digital bodily integrity is almost priceless.
- selectodude 18d ago
  Local frontier costs a half million dollars to run locally in anything higher than basically ternary.
  - txrx0000 18d ago
    Okay, that's technically true again, but local mid-tier like Qwen3.8-27B is only a year behind the closed frontier. I'm personally willing to be behind by a year if it gives me mental sovereignty against the big AI companies. They are extremely misaligned with me.
koito17 18d ago
GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary. In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passing apartment diagram pictures to Qwen and making it compare different apartments for which ones would feel the most spacious while optimizing for initial moving costs and other factors.) This was back in May and I wouldn't be surprised if there have been significant improvements since then. Overall, I think it's fair to compare a workflow like "use llama.cpp locally to upload some pictures and ask questions" to "open the ChatGPT app, upload pictures from your phone, and ask questions". Sure, you can't run a model like GPT-5.4 locally, but the model is mostly an implementation detail here. What a user will care about is: "when I go with the llama.cpp option, am I getting useful information from my conversations?"
- v3ss0n 18d ago
  Deepseek 4 flash can run locally , and qwen 3.8-next-flash , they are already gpt 5.6 tier.
- no-name-here 18d ago
  Wouldn't the better comparison still be against an AI provider with better privacy controls, especially if that's what someone cares about (even if they don't care about whether they're comparing a 35 billion param model vs a x trillion param model)?
  - koito17 18d ago
    Users generally have no way to verify that a third-party provider, even if they advertise themselves as privacy-focused, will adhere to their own terms. This is similar to the issue of privacy-focused VPN providers that claim to not log user activity (and then end up leaking user activity). You can get proof of ~P, but rarely proof of P, and often times the proof of ~P is due to police raids, data breaches, etc., not something of the provider's volition. What you can possibly audit is probably data sovereignty. For instance, I would not be surprised if Mistral's customers demand concrete evidence that their data is held within the European Union. But that is a distinct issue from training on input tokens.
- deleted 18d ago
  [deleted]
v3ss0n 18d ago
You haven't tried DeekSeek v4 or GLM 5.3 or Qwen 3.8 Next? You are missing out a lot. Try that with Hermes or Opencode or Deekseek Harness , even Qwen 3.8 27b works really well for that kind of that. I just ask it to install windows as a vm on my linux and install vs Community 2019 on it , and then build a legacy vb 2019 project on it. and sleep When i wake up : It installs Qemu , setup a vm , inside vm download and install windows 10 on its own , clicking next next next as needed , typing in things , writing powershell , python scripts , that run automatically after install by baking into CD that includes ssh server , reboot , it logins into ssh , trigger pythons script that continue installation of vs 2019 community , which includes a driver that click the installation steps , installs nuget , install all depedencies and then build the project into exe after i woke up. That is with 100% pure local AI .
- LargoLasskhyfv 18d ago
  Regarding DeepSeek, which I also like very much, have you tried https://reasonix.io https://reasonix.io ?
  - v3ss0n 18d ago
    Bot? Care to explain any difference vs DSH / OpenCode / Hermes ?
    - LargoLasskhyfv 18d ago
      Notbot! Are you unable to read, or what? Just fkn install it, and see how smooth it integrates?
- ASalazarMX 18d ago
  I've tried the latest Qwen, and without Internet, it still can go into an incoherent loop if you ask for, say, song lyrics. TBH the commercial models might do that too if not for their internal tooling.

txrx0000 18d ago

Technically true, but the delay between local and closed frontier is only a few months. And individual sovereignty / digital bodily integrity is almost priceless.

selectodude 18d ago
Local frontier costs a half million dollars to run locally in anything higher than basically ternary.
- txrx0000 18d ago
  Okay, that's technically true again, but local mid-tier like Qwen3.8-27B is only a year behind the closed frontier. I'm personally willing to be behind by a year if it gives me mental sovereignty against the big AI companies. They are extremely misaligned with me.

koito17 18d ago

GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary. In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passing apartment diagram pictures to Qwen and making it compare different apartments for which ones would feel the most spacious while optimizing for initial moving costs and other factors.) This was back in May and I wouldn't be surprised if there have been significant improvements since then. Overall, I think it's fair to compare a workflow like "use llama.cpp locally to upload some pictures and ask questions" to "open the ChatGPT app, upload pictures from your phone, and ask questions". Sure, you can't run a model like GPT-5.4 locally, but the model is mostly an implementation detail here. What a user will care about is: "when I go with the llama.cpp option, am I getting useful information from my conversations?"

v3ss0n 18d ago
Deepseek 4 flash can run locally , and qwen 3.8-next-flash , they are already gpt 5.6 tier.
no-name-here 18d ago
Wouldn't the better comparison still be against an AI provider with better privacy controls, especially if that's what someone cares about (even if they don't care about whether they're comparing a 35 billion param model vs a x trillion param model)?
- koito17 18d ago
  Users generally have no way to verify that a third-party provider, even if they advertise themselves as privacy-focused, will adhere to their own terms. This is similar to the issue of privacy-focused VPN providers that claim to not log user activity (and then end up leaking user activity). You can get proof of ~P, but rarely proof of P, and often times the proof of ~P is due to police raids, data breaches, etc., not something of the provider's volition. What you can possibly audit is probably data sovereignty. For instance, I would not be surprised if Mistral's customers demand concrete evidence that their data is held within the European Union. But that is a distinct issue from training on input tokens.
deleted 18d ago
[deleted]

v3ss0n 18d ago

You haven't tried DeekSeek v4 or GLM 5.3 or Qwen 3.8 Next? You are missing out a lot. Try that with Hermes or Opencode or Deekseek Harness , even Qwen 3.8 27b works really well for that kind of that. I just ask it to install windows as a vm on my linux and install vs Community 2019 on it , and then build a legacy vb 2019 project on it. and sleep When i wake up : It installs Qemu , setup a vm , inside vm download and install windows 10 on its own , clicking next next next as needed , typing in things , writing powershell , python scripts , that run automatically after install by baking into CD that includes ssh server , reboot , it logins into ssh , trigger pythons script that continue installation of vs 2019 community , which includes a driver that click the installation steps , installs nuget , install all depedencies and then build the project into exe after i woke up. That is with 100% pure local AI .

LargoLasskhyfv 18d ago
Regarding DeepSeek, which I also like very much, have you tried https://reasonix.io https://reasonix.io ?
- v3ss0n 18d ago
  Bot? Care to explain any difference vs DSH / OpenCode / Hermes ?
  - LargoLasskhyfv 18d ago
    Notbot! Are you unable to read, or what? Just fkn install it, and see how smooth it integrates?
ASalazarMX 18d ago
I've tried the latest Qwen, and without Internet, it still can go into an incoherent loop if you ask for, say, song lyrics. TBH the commercial models might do that too if not for their internal tooling.