3 ms·
"An example of this is DeepSeek-Coder, which can essentially be considered a fine-tune of a fine-tune of a Mixtral model" I've not heard that anywhere else. My
by simonw 2y ago
"An example of this is DeepSeek-Coder, which can essentially be considered a fine-tune of a fine-tune of a Mixtral model"
I've not heard that anywhere else. My impression from https://huggingface.co/deepseek-ai/deepseek-coder-33b-base https://huggingface.co/deepseek-ai/deepseek-coder-33b-base was that DeepSeek-Coder was trained from scratch.
- darksaints 2y agoThe current deepseek-coder version (v2) is actually a fine tune off of the deepseek v2 model, and was not trained from scratch. I’m now getting conflicting information about the origin of the deepseek MOE framework, so I may be wrong about it starting with a Mixtral model.