5 ms·

Exciting to see this being reproduced, loving the hyper-fast movement in open source! This is exactly why it is not “US vs China”, the battle is between heavil

Exciting to see this being reproduced, loving the hyper-fast movement in open source! This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source. Every believer in this tech owes DeepSeek some gratitude, but even they stand on shoulders of giants in the form of everyone else who pushed the frontier forward and chose to publish, rather than exploit, what they learned.

brianstrimp 2y ago
[dead]
raxxor 2y ago
Oh yes, I am firmly on Team China here because US companies got too greedy. Meta is an exception here though and they also propelled AI development massively. DeepSeek is awesome. Any AI task yet implemented in our business can be run from my local PC with just the smaller models. And my PC is fairly crappy to begin with. OpenAI looks quite silly with their "we have to close everything".
- manmal 2y ago
  Can you elaborate which models you are using? I‘m running an R1 distilled Qwen coder with 32B Q4, and while it’s giving useful answers, it‘s quite slow on my M1 Max. Slow enough that I keep reaching for cloud models.
  - raxxor 2y ago
    Not on my machine currently, I use the 14b Q4 model I think, which delivers very good answers. I run a 4060 with 16gb memory and performance is quite good. I used the largest model that was recommended with this amount of VRAM, I think it was the 14b one. I do have some applications that process images, text and pdf files and I use smaller models for extracting embeddings. I think my system wouldn't be able to handle it with decent speed otherwise. I do run LLM on a M1 16gb macbook air and performance is surprisingly good. Not for image synthesis though and a PC with a dedicated GPU is still significantly faster with LLM responses as well. Haven't tried to run deepseek on the macbook yet.
    - manmal 2y ago
      Interesting, I didn’t like the quality of the output of the 14B models. Could be the quantization though, apparently some are a bit broken.
- blackeyeblitzar 2y ago
  The US companies got too greedy? How? They invented this entire space, literally. DeepSeek built their base models off Llama releases and OpenAI outputs (or so it’s thought), and while they added some optimizations on top, it seems like they’ve lied about the costs to produce their models by simply being vague about their base model and training data, and quoting the cost of their final training run. And then there’s all the dystopian propaganda baked into these models, which threatens to misinform users at scale based on a government driven agenda. Hard to be on that team, let alone firmly, knowing that it’s giving power to a dictatorial regime.
  - wkat4242 2y ago
    The US models are also full of censorship. For example the US is much more sensitive to anything related to sexuality and here in Europe it's quite frustrating to deal with that censorship.
    - infecto 2y ago
      I think we will find that each region will have their own flair of censorship. The only reason it stands out more from a Chinese perspective is the requirement to have alignment with PRC/CCP rhetoric.
      - wkat4242 2y ago
        Yes that's what I mean. I wish all models were uncensored and it would just be up to the implementer to decide how to finetune on top of that. Save for the super crazy stuff of course.
  - timeon 2y ago
    > DeepSeek built their base models off Llama releases and OpenAI outputs Those models are also trained on data that was ignoring licenses / copyrighted content.
    - ithkuil 2y ago
- nejsjsjsbsb 2y ago
  I'm on team open source. To me the exciting thing was ollama downloading the 7B and running it on a 5yo cheap lonovo and getting a token rate similar to the first release of ChatGPT. Running local on CPU opens so much possibilities for smart and privacy focused home devices that serve you. In my test it hallucinated confidently but my interest is in simple second brain like rag. "Hey thingy, what is my schedule today?" Need it to be a bit faster though as the thinking part adds a lot of latency.
  - raxxor 2y ago
    The thinking is quite fascinating though, I love reading it. Especially when it notices something must be wrong. It will probably be very helpful to refine answer for itself and other models. It does add latency of course, but I still think that I could provide all AI needs of my company (industrial production) with a simple older off the shelf PC. My GPU is decently recent, but the smallest model of the series and otherwise the machine is a rusty bucket. I didn't test it thoroughly yet, but I have some invoices where I need to extract info and it did a perfect job until now. But I don't think there is any LLM yet that can do that without someone checking the output.
jillesvangurp 2y ago
China is merely the largest of a wide array of entities that may not necessarily like the status quo of Silicon Valley being our tech overlords. There are plenty of places with bright people. Easy to say because a lot of them immigrate to California. But of course the places they come from (China, Europe, India, Russia etc.) have ambitions as well. You'll find natives of each of those in the likes of OpenAI, Google, Microsoft, etc. And quite often at executive levels even. Silicon Valley has mo moat other than money. It kind of runs on openness and freedom of movement of people. Companies constantly poach people from each other. And there's a constant movement of people (and knowledge) in and out of the area. Money is what attracts these people and keeps them there for a while. But of course that status quo was upset a little bit with VCs turning into penny pinching misers lately and lockdowns proving (to them) that it was cheaper to host your tech teams remotely. Which means knowledge is now more distributed than it used to be. So, it's not surprising that people outside of Silicon Valley are not waiting patiently for OpenAI to do whatever it is they are doing in between having moral existential crises, trying to oust their CEO, pontificating about AGIs, etc. They are taking things into their own hands. The brute force / VC funding driven approach that OpenAI has used yielded massive results in the last few years. But ever since Meta opensourced their models, OSS models and optimizations have been catching up. On a hardware resource usage basis, these models started to outperform their bigger peers last year and now the game is up for the training process as well. Meaning they get better results for the same money. A major hurdle here was the model training process. Which the Chinese seem to have proven can be massively optimized as well. Cutting cost by a few orders of magnitude is a big deal. And at the same time doing the same thing at larger scale (aka. throwing more money at the problem) seems to have diminishing returns. Until that changes, that means the playing field has somewhat leveled now. That's a good thing.
MagicMoonlight 2y ago
I like that it’s open source, but ultimately it is china so we can’t trust it. It’s trivial to implement bias in models (hence the no-no filters in chatgpt) so if they’re smart they’ll do what they do with tiktok and make the answers different for their rivals.
- esperent 2y ago
  The thing about open source is that you don't need to trust it. They shared their methodology, so if they are legit, someone else will reproduce what they did very quickly. I expect Meta, Amazon, Google, and Anthropic are on the case right now. From this list, the only one I trust any more than I trust Deepseek is Anthropic. The other three have shown they'll instantly bend the knee to whomever is in power, and that's exactly the same thing people are worried that Deepseek is doing. Last year I would have said I trusted American companies more than Chinese. But last year feels like a long time ago.
  - _t9ow 2y ago
    > the only one I trust any more than I trust Deepseek is Anthropic Will you kindly explain why? Thank you.
    - frabcus 2y ago
      Anthropic was the only company in that list not to have paid for their CEO or founder to attend the inaugeration of the current ruler of the exectuive branch of the US a week ago.
      - _t9ow 2y ago
        That makes sense, but why trust them more than DeepSeek?
        esperent 2y ago
        Because, from my non-expert but reasonably well informed understanding of world affairs, there's a very high chance of a Chinese company in an important area like AI having to bend the knee to Xi Jinping pretty much exactly as the American companies are doing with Trump.
bgnn 2y ago
And in this case it's heavily capitalized open source, which scares the techbros.
LunaSea 2y ago
> This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source. Ah yes the "open source" code that was not released by the DeepSeek team and the tens of thousands of professional grade GPUs that were contributed by the "community". DeepSeek is based on Llama which was produced by ... Meta.
- Palmik 2y ago
  DeepSeek v3/r1 isn't based on llama architecture. It uniquely combines and contributes several novel approaches. Meta never released a mixture of expert model (they failed to train a good one, according to reliable rumors). And MoE is just one of few ingredients that make DeepSeek v3/R1 interesting and good.
typ 2y ago
Open source has pragmatic merits and I love the culture. But I don't like associating it with a moral high ground because it doesn't charge you money. By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. Not all business sectors can afford to sustain with the Open Source model.
- mordae 2y ago
  > By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. You aren't? You're weird.
  - nejsjsjsbsb 2y ago
    Yeah. Actually if those companies did I suspect it wouldn't hurt them one iota.
- nejsjsjsbsb 2y ago
  The freedom is mostly not about the money. The 3D model Benchy was free, while being not free as people found out. Luckily the copyright owners treated it like people were free and free for now... but that could change.
- anthk 2y ago
  - Talos Workstations - Libre engines such as the ones from https://osgameclones.com https://osgameclones.com - Godot - Use anything else, there are tons of libre search engines

brianstrimp 2y ago

[dead]

raxxor 2y ago

Oh yes, I am firmly on Team China here because US companies got too greedy. Meta is an exception here though and they also propelled AI development massively. DeepSeek is awesome. Any AI task yet implemented in our business can be run from my local PC with just the smaller models. And my PC is fairly crappy to begin with. OpenAI looks quite silly with their "we have to close everything".

manmal 2y ago
Can you elaborate which models you are using? I‘m running an R1 distilled Qwen coder with 32B Q4, and while it’s giving useful answers, it‘s quite slow on my M1 Max. Slow enough that I keep reaching for cloud models.
- raxxor 2y ago
  Not on my machine currently, I use the 14b Q4 model I think, which delivers very good answers. I run a 4060 with 16gb memory and performance is quite good. I used the largest model that was recommended with this amount of VRAM, I think it was the 14b one. I do have some applications that process images, text and pdf files and I use smaller models for extracting embeddings. I think my system wouldn't be able to handle it with decent speed otherwise. I do run LLM on a M1 16gb macbook air and performance is surprisingly good. Not for image synthesis though and a PC with a dedicated GPU is still significantly faster with LLM responses as well. Haven't tried to run deepseek on the macbook yet.
  - manmal 2y ago
    Interesting, I didn’t like the quality of the output of the 14B models. Could be the quantization though, apparently some are a bit broken.
blackeyeblitzar 2y ago
The US companies got too greedy? How? They invented this entire space, literally. DeepSeek built their base models off Llama releases and OpenAI outputs (or so it’s thought), and while they added some optimizations on top, it seems like they’ve lied about the costs to produce their models by simply being vague about their base model and training data, and quoting the cost of their final training run. And then there’s all the dystopian propaganda baked into these models, which threatens to misinform users at scale based on a government driven agenda. Hard to be on that team, let alone firmly, knowing that it’s giving power to a dictatorial regime.
- wkat4242 2y ago
  The US models are also full of censorship. For example the US is much more sensitive to anything related to sexuality and here in Europe it's quite frustrating to deal with that censorship.
  - infecto 2y ago
    I think we will find that each region will have their own flair of censorship. The only reason it stands out more from a Chinese perspective is the requirement to have alignment with PRC/CCP rhetoric.
    - wkat4242 2y ago
      Yes that's what I mean. I wish all models were uncensored and it would just be up to the implementer to decide how to finetune on top of that. Save for the super crazy stuff of course.
- timeon 2y ago
  > DeepSeek built their base models off Llama releases and OpenAI outputs Those models are also trained on data that was ignoring licenses / copyrighted content.
  - ithkuil 2y ago
nejsjsjsbsb 2y ago
I'm on team open source. To me the exciting thing was ollama downloading the 7B and running it on a 5yo cheap lonovo and getting a token rate similar to the first release of ChatGPT. Running local on CPU opens so much possibilities for smart and privacy focused home devices that serve you. In my test it hallucinated confidently but my interest is in simple second brain like rag. "Hey thingy, what is my schedule today?" Need it to be a bit faster though as the thinking part adds a lot of latency.
- raxxor 2y ago
  The thinking is quite fascinating though, I love reading it. Especially when it notices something must be wrong. It will probably be very helpful to refine answer for itself and other models. It does add latency of course, but I still think that I could provide all AI needs of my company (industrial production) with a simple older off the shelf PC. My GPU is decently recent, but the smallest model of the series and otherwise the machine is a rusty bucket. I didn't test it thoroughly yet, but I have some invoices where I need to extract info and it did a perfect job until now. But I don't think there is any LLM yet that can do that without someone checking the output.

jillesvangurp 2y ago

China is merely the largest of a wide array of entities that may not necessarily like the status quo of Silicon Valley being our tech overlords. There are plenty of places with bright people. Easy to say because a lot of them immigrate to California. But of course the places they come from (China, Europe, India, Russia etc.) have ambitions as well. You'll find natives of each of those in the likes of OpenAI, Google, Microsoft, etc. And quite often at executive levels even. Silicon Valley has mo moat other than money. It kind of runs on openness and freedom of movement of people. Companies constantly poach people from each other. And there's a constant movement of people (and knowledge) in and out of the area. Money is what attracts these people and keeps them there for a while. But of course that status quo was upset a little bit with VCs turning into penny pinching misers lately and lockdowns proving (to them) that it was cheaper to host your tech teams remotely. Which means knowledge is now more distributed than it used to be. So, it's not surprising that people outside of Silicon Valley are not waiting patiently for OpenAI to do whatever it is they are doing in between having moral existential crises, trying to oust their CEO, pontificating about AGIs, etc. They are taking things into their own hands. The brute force / VC funding driven approach that OpenAI has used yielded massive results in the last few years. But ever since Meta opensourced their models, OSS models and optimizations have been catching up. On a hardware resource usage basis, these models started to outperform their bigger peers last year and now the game is up for the training process as well. Meaning they get better results for the same money. A major hurdle here was the model training process. Which the Chinese seem to have proven can be massively optimized as well. Cutting cost by a few orders of magnitude is a big deal. And at the same time doing the same thing at larger scale (aka. throwing more money at the problem) seems to have diminishing returns. Until that changes, that means the playing field has somewhat leveled now. That's a good thing.

MagicMoonlight 2y ago

I like that it’s open source, but ultimately it is china so we can’t trust it. It’s trivial to implement bias in models (hence the no-no filters in chatgpt) so if they’re smart they’ll do what they do with tiktok and make the answers different for their rivals.

esperent 2y ago
The thing about open source is that you don't need to trust it. They shared their methodology, so if they are legit, someone else will reproduce what they did very quickly. I expect Meta, Amazon, Google, and Anthropic are on the case right now. From this list, the only one I trust any more than I trust Deepseek is Anthropic. The other three have shown they'll instantly bend the knee to whomever is in power, and that's exactly the same thing people are worried that Deepseek is doing. Last year I would have said I trusted American companies more than Chinese. But last year feels like a long time ago.
- _t9ow 2y ago
  > the only one I trust any more than I trust Deepseek is Anthropic Will you kindly explain why? Thank you.
  - frabcus 2y ago
    Anthropic was the only company in that list not to have paid for their CEO or founder to attend the inaugeration of the current ruler of the exectuive branch of the US a week ago.
    - _t9ow 2y ago
      That makes sense, but why trust them more than DeepSeek?
      - esperent 2y ago
        Because, from my non-expert but reasonably well informed understanding of world affairs, there's a very high chance of a Chinese company in an important area like AI having to bend the knee to Xi Jinping pretty much exactly as the American companies are doing with Trump.

bgnn 2y ago

And in this case it's heavily capitalized open source, which scares the techbros.

LunaSea 2y ago

> This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source. Ah yes the "open source" code that was not released by the DeepSeek team and the tens of thousands of professional grade GPUs that were contributed by the "community". DeepSeek is based on Llama which was produced by ... Meta.

Palmik 2y ago
DeepSeek v3/r1 isn't based on llama architecture. It uniquely combines and contributes several novel approaches. Meta never released a mixture of expert model (they failed to train a good one, according to reliable rumors). And MoE is just one of few ingredients that make DeepSeek v3/R1 interesting and good.

typ 2y ago

Open source has pragmatic merits and I love the culture. But I don't like associating it with a moral high ground because it doesn't charge you money. By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. Not all business sectors can afford to sustain with the Open Source model.

mordae 2y ago
> By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. You aren't? You're weird.
- nejsjsjsbsb 2y ago
  Yeah. Actually if those companies did I suspect it wouldn't hurt them one iota.
nejsjsjsbsb 2y ago
The freedom is mostly not about the money. The 3D model Benchy was free, while being not free as people found out. Luckily the copyright owners treated it like people were free and free for now... but that could change.
anthk 2y ago
- Talos Workstations - Libre engines such as the ones from https://osgameclones.com https://osgameclones.com - Godot - Use anything else, there are tons of libre search engines