3 ms·

I'm testing it for system programming brainstorming, code reviews and Python test units writing, and my impression is that it's a Sonnet 3.5 level model for mos

I'm testing it for system programming brainstorming, code reviews and Python test units writing, and my impression is that it's a Sonnet 3.5 level model for most tasks. I said a few things here: https://www.youtube.com/watch?v=xjCqi9JK440 https://www.youtube.com/watch?v=xjCqi9JK440 but in general this is really an open weights frontier model, the first one that we get (IMHO llama 3.1 405B does not fit the definition, and the actual model quality is far from the benchmarks). Also the extreme inference speed due to MoE and other design choices improves the user experience a lot. I also tested asking questions with very large contexts (PDFs, large C files) at play, and it performs very well. Also don't just focus on this model but check out what DeepSeek mission is, and the CEO words in the recently released interview. They want to be the DJI / Bambulab of AI, basically: leaders and not followers, and after V3 it's hard to say they don't have the right brains to do that.

maeil 2y ago
> IMHO llama 3.1 405B does not fit the definition, and the actual model quality is far from the benchmarks Very true. Meta has been disappointing so far, and it takes away from the blog that it starts with a graph of a completely misrepresentative benchmark (MMLU) that shows things like Llama3.1-405b besting Gemini 1.5-pro, 4o-mini above Haiku 3.5, and so on. But all this means is that the leap in terms of open weights model is even far bigger with Deepseek v3.
knicholes 2y ago
Do you mind sharing your workflow with how you use this model? Is it integrated with an IDE, are you using the command line, maybe a UI with each question then copy/pasting the results?
- antirez 2y ago
  Sure! I'll do several videos in the future with my workflow. No ide, I just use the web interface.
cheema33 2y ago
This has exactly my experience. I use all the main models almost every day. I switch between them several times a day and compare and contrast their output. Deepseek is shockingly good! How they got there with a relatively small budget is mind blowing.

maeil 2y ago

> IMHO llama 3.1 405B does not fit the definition, and the actual model quality is far from the benchmarks Very true. Meta has been disappointing so far, and it takes away from the blog that it starts with a graph of a completely misrepresentative benchmark (MMLU) that shows things like Llama3.1-405b besting Gemini 1.5-pro, 4o-mini above Haiku 3.5, and so on. But all this means is that the leap in terms of open weights model is even far bigger with Deepseek v3.

knicholes 2y ago

Do you mind sharing your workflow with how you use this model? Is it integrated with an IDE, are you using the command line, maybe a UI with each question then copy/pasting the results?

antirez 2y ago
Sure! I'll do several videos in the future with my workflow. No ide, I just use the web interface.

cheema33 2y ago

This has exactly my experience. I use all the main models almost every day. I switch between them several times a day and compare and contrast their output. Deepseek is shockingly good! How they got there with a relatively small budget is mind blowing.