6 ms·

Can you explain to a non ML software engineer what these distillation methods mean? What does it mean to have R1 train a Llama model? What is special about Deep

Can you explain to a non ML software engineer what these distillation methods mean? What does it mean to have R1 train a Llama model? What is special about DeepSeek’s distillation methods? Thanks!

littlestymaar 2y ago
Submit a bunch of prompts to Deepseek R1 (a few tens of thousands), and then do a full fine tuning of the target model on the prompt/response pair.
dcre 2y ago
Distilling means fine-tuning an existing model using outputs from the bigger model. The special technique is in the details of what you choose to generate from the bigger model, how long to train for, and a bunch of other nitty gritty stuff I don’t know about because I’m also not an ML engineer. Google it!
- lr1970 2y ago
  > Distilling means fine-tuning an existing model using outputs from the bigger model. Crucially, the output of the teacher model includes token probabilities so that the fine-tuning is trying to learn the entire output distribution.
  - numba888 2y ago
    That's possible only if they use the same tokens. Which likely requires they share the same tokenizer. Not sure that's the case here, R1 was built on OpenAI closed model's output.
    - anon373839 2y ago
      That was an (as far as I can tell) unsubstantiated claim made by OpenAI. It doesn’t even make sense, as o1’s reasoning traces are not provided to the user.
      - int_19h 2y ago
        One reason to believe OpenAI here is that R1 will occasionally claim to be made by OpenAI, which in e.g. LLaMA finetunes is indicative of using synthetic data generated by ChatGPT. Note that this isn't necessarily o1. While o1 is specifically trained to do CoT, you can also make 4o etc produce it with the appropriate prompts, and then train on that output.
        HPsquared 2y ago
        I suppose it might be hard to avoid encountering ChatGPT outputs "in the wild" now, even if they don't explicitly use it for training material.
- sumedh 2y ago
  > Google it! Or you could provide some example links
  - dcre 2y ago
    This only makes sense if I have some great canonical explanation of distillation on hand. But it’s a simple concept. There are hundreds of identical explanations online.
    - sumedh 2y ago
      Are all the 100 explanations good or would you recommend one of them?
andix 2y ago
Its llama/quen with some additional training to add reasoning. In a similar way deep seeks v3 was trained into r1. It also looks to me like there was some Chinese propaganda trained into llama/quen too, but that’s just my observation.
- kvirani 2y ago
  You have my curiosity. Like what and how did you find it?
  - andix 2y ago
    Ask about Xi Jinping in all the ways you can imagine (jokes, bad habits, failures, embarrassing facts, ...). Compare the responses to other well known politicians, use the same prompt in a fresh conversation with a different name. Ask about the political system of china and its flaws. Compare the sentiment of the responses with answers about other political systems. You might get some critical answers, but the sentiment is usually very positive towards china. Sometimes it doesn't even start reasoning and directly spits out propaganda, that doesn't even answer your question. You can't test it with deep seek dot com, because it will just remove the answers on those "sensitive" topics. I've mostly tested with 7b from ollama. You might experience something like that with 1.5b too, but 1.5b barely works at all.
    - mdp2021 2y ago
      Could it be just a bias inside the selected training material?
      - andix 2y ago
        Feel free to call propaganda a bias if you like. But if it walks like a duck, quacks like a duck, ...
        mdp2021 2y ago
        This is HN: my focus is technical (here specifically), maybe "technical" in world assessment and future prediction (in other pages). I.e.: I am just trying to understand the facts.
corysama 2y ago
"Quantized" models try to approximate the full model using less bits. "Distilled" models are other models (Llama, Qwen) that have been put through an additional training round using DeepSeek as a teacher.
- beefnugs 2y ago
  What is the effect of this less bits? Is it like truncating hashes where you start going off into the wrong thing entirely, or more like less accuracy so that if you are talking about soft penguins it will start thinking you mean wet penguins? And is there a domain specific term I can look into if I wanted to read about someone trying to keep all the bits, but the runtime (trying to save ram) focusing in on parts of the data instead of this quantization?
  - corysama 2y ago
    I'm pretty far from an expert. But, at it's core ML is a bunch of matrix multiplications glued together with non-linear functions. So, quantization leads to less accuracy in the matrices of weights. Not, changes in hashes where 1 wrong bit is meaningless. The folks who quantized DeepSeek say they used a piece of tech called "BitsAndBytes". https://unsloth.ai/blog/dynamic-4bit https://unsloth.ai/blog/dynamic-4bit Googling around for "bitsandbytes ai quantization" turns up this article which looks nice https://generativeai.pub/practical-guide-of-llm-quantization-gptq-awq-bitsandbytes-and-unsloth-bdeaa2c0bbf6 https://generativeai.pub/practical-guide-of-llm-quantization...

Can you explain to a non ML software engineer what these distillation methods mean? What does it mean to have R1 train a Llama model? What is special about DeepSeek’s distillation methods? Thanks!

littlestymaar 2y ago

Submit a bunch of prompts to Deepseek R1 (a few tens of thousands), and then do a full fine tuning of the target model on the prompt/response pair.

dcre 2y ago

Distilling means fine-tuning an existing model using outputs from the bigger model. The special technique is in the details of what you choose to generate from the bigger model, how long to train for, and a bunch of other nitty gritty stuff I don’t know about because I’m also not an ML engineer. Google it!

lr1970 2y ago
> Distilling means fine-tuning an existing model using outputs from the bigger model. Crucially, the output of the teacher model includes token probabilities so that the fine-tuning is trying to learn the entire output distribution.
- numba888 2y ago
  That's possible only if they use the same tokens. Which likely requires they share the same tokenizer. Not sure that's the case here, R1 was built on OpenAI closed model's output.
  - anon373839 2y ago
    That was an (as far as I can tell) unsubstantiated claim made by OpenAI. It doesn’t even make sense, as o1’s reasoning traces are not provided to the user.
    - int_19h 2y ago
      One reason to believe OpenAI here is that R1 will occasionally claim to be made by OpenAI, which in e.g. LLaMA finetunes is indicative of using synthetic data generated by ChatGPT. Note that this isn't necessarily o1. While o1 is specifically trained to do CoT, you can also make 4o etc produce it with the appropriate prompts, and then train on that output.
      - HPsquared 2y ago
        I suppose it might be hard to avoid encountering ChatGPT outputs "in the wild" now, even if they don't explicitly use it for training material.
sumedh 2y ago
> Google it! Or you could provide some example links
- dcre 2y ago
  This only makes sense if I have some great canonical explanation of distillation on hand. But it’s a simple concept. There are hundreds of identical explanations online.
  - sumedh 2y ago
    Are all the 100 explanations good or would you recommend one of them?

andix 2y ago

Its llama/quen with some additional training to add reasoning. In a similar way deep seeks v3 was trained into r1. It also looks to me like there was some Chinese propaganda trained into llama/quen too, but that’s just my observation.

kvirani 2y ago
You have my curiosity. Like what and how did you find it?
- andix 2y ago
  Ask about Xi Jinping in all the ways you can imagine (jokes, bad habits, failures, embarrassing facts, ...). Compare the responses to other well known politicians, use the same prompt in a fresh conversation with a different name. Ask about the political system of china and its flaws. Compare the sentiment of the responses with answers about other political systems. You might get some critical answers, but the sentiment is usually very positive towards china. Sometimes it doesn't even start reasoning and directly spits out propaganda, that doesn't even answer your question. You can't test it with deep seek dot com, because it will just remove the answers on those "sensitive" topics. I've mostly tested with 7b from ollama. You might experience something like that with 1.5b too, but 1.5b barely works at all.
  - mdp2021 2y ago
    Could it be just a bias inside the selected training material?
    - andix 2y ago
      Feel free to call propaganda a bias if you like. But if it walks like a duck, quacks like a duck, ...
      - mdp2021 2y ago
        This is HN: my focus is technical (here specifically), maybe "technical" in world assessment and future prediction (in other pages). I.e.: I am just trying to understand the facts.

corysama 2y ago

"Quantized" models try to approximate the full model using less bits. "Distilled" models are other models (Llama, Qwen) that have been put through an additional training round using DeepSeek as a teacher.

beefnugs 2y ago
What is the effect of this less bits? Is it like truncating hashes where you start going off into the wrong thing entirely, or more like less accuracy so that if you are talking about soft penguins it will start thinking you mean wet penguins? And is there a domain specific term I can look into if I wanted to read about someone trying to keep all the bits, but the runtime (trying to save ram) focusing in on parts of the data instead of this quantization?
- corysama 2y ago
  I'm pretty far from an expert. But, at it's core ML is a bunch of matrix multiplications glued together with non-linear functions. So, quantization leads to less accuracy in the matrices of weights. Not, changes in hashes where 1 wrong bit is meaningless. The folks who quantized DeepSeek say they used a piece of tech called "BitsAndBytes". https://unsloth.ai/blog/dynamic-4bit https://unsloth.ai/blog/dynamic-4bit Googling around for "bitsandbytes ai quantization" turns up this article which looks nice https://generativeai.pub/practical-guide-of-llm-quantization-gptq-awq-bitsandbytes-and-unsloth-bdeaa2c0bbf6 https://generativeai.pub/practical-guide-of-llm-quantization...