6 ms·
Naive question from a bystander , but since DeepSeek is open source and is on par with o1-pro (is it?), shouldn't we expect that anybody with the computer power
by yodsanklai 2y ago
Naive question from a bystander , but since DeepSeek is open source and is on par with o1-pro (is it?), shouldn't we expect that anybody with the computer power is capable to compete with o1-pro?
- golol 2y agoDeepseek is not on par with o1.
- resters 2y agoYes it is!
- jamalaramala 2y agoIt probably depends on the benchmark you choose; according to Chatbot Arena, Deepseek-R1 ranks similarly to o1-2024-12-17; and Grok3 is just 3% above these models in "Arena Score" points.
- golol 2y agoChatbot Arena is not really a great benchmark imo
- menaerus 2y agoYou'd still need a fairly large amount of compute power to be able to run DeepSeek R1 locally, no?
- roblabla 2y agoWell yes, but not so large that it's completely prohibitive. People have been running the full models on computers going as low as $6000: https://x.com/carrigmat/status/1884244369907278106 https://x.com/carrigmat/status/1884244369907278106 Of course this is for a personal instance, you'd need a much more expensive setup to handle concurrent users. And that's to run it, not train it.
- plagiarist 2y agoSortof a letdown that after 24 32Gb RAM sticks you only get 6-8 tokens per second.
- menaerus 2y ago6k is not that bad considering that top of the line Apple laptop costs as much. However, I don't have X so unfortunately I can't read the details.
- longitudinal93 2y agoYou can read the whole thread through nitter: https://xcancel.com/carrigmat/status/1884244369907278106 https://xcancel.com/carrigmat/status/1884244369907278106
- fspeech 2y agoA better approach is to split the model with MOEs running on CPUs and MLAs running on GPU. See the ktransformers project: https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/DeepseekR1_V3_tutorial.md https://github.com/kvcache-ai/ktransformers/blob/main/doc/en... This takes advantage of the sparsity of MOE and the efficient KV-cache of MLA.
- menaerus 2y agoYou perhaps forgot to mention that for their AMX optimizations to be even feasible you'd need to spend ~$10k for a single CPU, let alone the whole system which is probably ~$100k.
- tucnak 2y ago> DeepSeek is open source and is on par with o1-pro (is it?) There is no being "on par" in this space. Model providers are still mostly optimising for a handful of benchmarks / goals, like we can already see that Grok 3 is doing incredibly well on human preference (LM Arena) however with Style Control, it's suddenly behind ChatGPT-4o-latest and Gemini 2.0 is out the picture. So even within a single domain, goal, benchmark—it's not as straightforward as to say that one model is "on par" with another. > shouldn't we expect that anybody with the computer power is capable to compete with o1-pro? Not necessarily. I know it may be tempting to think that Grok 3 is entirely a result of xAI having lots of "computer power", but you have to recognise that this mindset is coming from a place of ignorance, not wisdom. Moreover, it doesn't even pass off as "cynical" view, because it's common knowledge that model training is really, really complicated. DeepSeek results are note-worthy, and really influential in some respects, but it hasn't magically "solved" training, or made training necessarily easier / less expensive for the interested parties. They never shared the low-level performance improvements, just model weights and lots of insight. For talented researchers, this is valuable, of course, but it's not like "anybody" could easily benefit from it in their training regimes. Update: RFT (contra SFT) is becoming really popular with service providers, and it's not been "standardised" beyond whatever reproductions to have emerged in the weeks prior, moreover R1 cost is still pretty high[1] at something like $7/Mtok, & bandwidth is really not great. Consider something like Google Vertex AI's batch pricing for Gemini 1.5 Pro and Gemini 2.0 Flash models, which is at 50% discount, and their prompt caching which is at 75% discount. R1 is still got a way to go. [1]: https://openrouter.ai/deepseek/deepseek-r1/providers?sort=throughput https://openrouter.ai/deepseek/deepseek-r1/providers?sort=th...
- dtquad 2y agoThe full-sized DeepSeek-R1 is on par with o1. o1-pro is "o1 on steroids" and was the first selling point of the $200/month Pro subscription but they later also added "Deep Research" and Operator to the Pro subscription.
- guax 2y agoEvery year seems like we get worse at naming things in non confusing ways. I am waiting for the o1-pro-max now, pro max ultra and pro max ultra plus.
- theptip 2y agoIt’s hard because there are multiple dimensions being upgraded at different cadences. Architecture, parameter count, etc.
- polski-g 2y agoI was recently laid off from OpenAI. My job was coming up with names for their models.
- jkestner 2y agoThen you’ll know what the naming AI was trained on.
- AlchemistCamp 2y ago
- ritz_labringue 2y agoIt's not on par with o1, let alone o1-pro
- nwienert 2y agoIt’s pretty on par with o1, better at many coding questions.
- flir 2y agoI found it better at reasoning, worse at coding. Not doubting your experience, just thinking how subjective it all is.
- csomar 2y agoIt's on par/better/worse depending on the problem. o1 is significantly worse, for example, in Rust programming than Claude 3.5; at least for me.
- gopher_space 2y agoClaude really likes producing code, that’s for sure. I feel like it’s a useful tool once I’ve deconstructed a project past a certain point.
- kragen 2y agoNo DeepSeek model is open source; they're freely redistributable, but without source.
- samsepi01 2y agoI guess when it comes to LLM's what is considered the "source" - the weights or the code used to build the weights?
- kragen 2y agoTo the extent that the concept is applicable, it would be the training data and the training code.
- Grimblewald 2y agoYou're forgetting the flour in your cake recipie, the data, arguably the singularly most important part.