14 ms·
An analysis of DeepSeek's R1-Zero and R1
- bandrami 2y agoJust because I'm an incurable cynic, has anybody run wireshark on it and checked that it actually does process entirely offline?
- dagelf 2y agoWell o3 scored 75% on AGI-1, R1 and o1 only 25%.... watch this space though....
- mohsen1 2y agowith 57 million(!!) tokens
- sheepdestroyer 2y agoFrom the article : o3 (low) 75.7% 335K $20 o3 (high) 87.5% 57M $3.4K
- jl6 2y ago$3.4K is about what you might pay a magic circle lawyer for an opinion on a matter. Not saying o3 is an efficient use of resources, just saying that it’s not outlandish that a sufficiently good AI could be worth that kind of money.
- ant6n 2y agoWhat’s the liability insurance of the AI like
- baq 2y agoRefer to IBM’s 1979 slide for details on that
- victorbjorklund 2y agoYou pay that price to a law firm to get good service and to get a "guarantee" of correctness. You get neither from an LLM. Not saying it is not worth anything but you cant compare it to a top law firm.
- nl 2y agoYou absolutely do not get a "guarantee" of correctness (event with the airquotes) from any lawyer.
- manquer 2y agoYou can sue a lawyer giving certain kinds of bad advice and occasionally win . That is what the guarantee is about
- bobxmax 2y agoYou can probably sue Open AI for getting bad legal advice from ChatGPT too.
- throw-qqqqq 2y agoSure, but can you also win the case ;)? On the bottom of ChatGPT.com I see a disclaimer: “ChatGPT can make mistakes. Check important info”. I don’t think you can succesfully sue with such caveat emptor.
- mrandish 2y agoWhen I saw these numbers back in the initial o3-ARC post, I immediately converted them into "$ per ARC-AGI-1 %" and concluded we may be at a point where each increased increment of 'real human-like novel reasoning' gets exponentially more compute costly. If Mike Knoop is correct, maybe R1 is pointing the way toward more efficient approaches. That would certainly be a good thing. This whole DeepSeek release and the reactions have shown by limiting the export to China of high-end GPUs, the US incentivized China to figure out how to make low-end GPUs work really well. The more subtle meta-lesson here is that the massive flood of investment capital being shoved toward leading edge AI companies has fostered a drag race mentality which prioritized winning top-line performance far above efficiency, costs, etc.
- Davidzheng 2y agoI view it as a positive that the methodology can take in more compute (bitter lesson style)
- optimalsolver 2y agoBut can o3 write a symphony? Seriously though, I'd like to hear suggestions on how to automatically evaluate an AI model's creativity, no humans in the loop.
- fragmede 2y agowe'd have to create a numerical scale for creativity, from boring to Dali, with milliEschers and MegaGeigers somewhere in there as well
- rpastuszak 2y agoIt's essential that we quantify everything so that we can put a price on it. I'd go with Kahlograms though.
- johnfn 2y agoHave you tried suno.ai?
- baq 2y agoLLMs have read everything humans made so just ask one if there’s anything truly new in that freshly confabulated slop-phony.
- gsam 2y agoIn my view there's two modes of creativity: 1. That two distant topics or ideas are actually much more closely related. The creative sees one example of an idea and applies it to a discipline that nobody expects. In theory, reduction of the maximally distant can probably be measured with a tangible metric. 2. Discovery of ideas that are even more maximally distant. Pushing the edge, and this can be done by pure search and randomness actually. But it's no good if it's garbage. The trick is, what is garbage? That is very context dependent. (Also, a creative might be measured on the efficiency of these metrics rather than absolute output)
- levocardia 2y agoWhat's interesting is that you can already see the "AI race" dynamics in play -- OpenAI must be under immense market pressure to push o3 out to the public to reclaim "king of the hill" status.
- spoaceman7777 2y agoI suppose they're under some pressure to release o3-mini, since r1 is roughly a peer for that, but r1 itself is still quite rough. The o1 series had seen significantly more QA time to smooth out the rough edges, and idiosyncracies what a "production" model should be optimized for, vs. just a top scorer on benchmarks. We'll likely only see o3 once there is a true polished peer for it. It's a race, and companies are keeping their best models close to their chest, as they're used internally to train smaller models. e.g., Claude 3.5 Opus has been around for quite a while, but it's unreleased. Instead, it was just used to refine Claude Sonnet 3.5 into Claude Sonnet 3.6 (3.6 is for lack of a better name, since it's still called 3.5). We also might see a new GPT-4o refresh trained up using GPT-o3 via deepseek's distillation technique and other tricks. There are a lot of new directions to go in now for OpenAI, but unfortunately, we won't likely see them until their API dominance comes under threat.
- danenania 2y agoThat could also definitely make sense if the SOTA models are too slow and expensive to be popular with a general audience.
- amelius 2y agoYeah, but they can use DeepSeek's new algorithm too.
- anothermathbozo 2y agoThe claim is that this removes the human bottleneck (aka SFT or supervised fine tuning) on domains with a verifiable reward. Critically, this verifiable reward is extremely hard to pin down in nearly all domains besides mathematics and computer science.
- Onavo 2y agoBy verifiable do they mean it in the complexity theory P/NP sense of the word?
- ks2048 2y agoI think it just means that you can objectively score an answer as being correct or not. (e.g. if the generated program passes some tests; a discovered proof is valid, etc).
- pertymcpert 2y agoThey mean that the solutions can be verified to be correct in a binary sense. E.g. a coding solution passes all the unit tests vs writing poetry.
- logicchains 2y agoAs in there's an objective truth that can be determined by a computer. E.g. whether code compiles, whether a unit test passes, whether the answer given to a mathematical question like 3+5 is correct. Many other fields have no objective truth (like art or creative writing), or objective truth requires measurement of the physical world (although if the world can be simulated accurately enough for the problem class at hand, then sufficient training data can still be generated by a computer).
- bpfrh 2y agoIsn't "code compiles" an insufficient criteria? e.g you would need to prove that for all inputs the code produces the correct output which would in turn make the problem way more complex
- artninja1988 2y ago>Ultimately, R1-Zero demonstrates the prototype of a potential scaling regime with zero human bottlenecks – even in the training data acquisition itself. I would like this to be true, but doesn't the way they're doing RL also require tons of human data?
- Davidzheng 2y agoI think yes. But hopefully in math with compute advances we can lower the human data input by increasing the gap that is bridged by raw model capabilities vs search augmentation (either with tree search or full rollouts)
- polishdude20 2y agoI predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests and fixing the code. Then, it deploys the software in production for you. It compiles your app to an APK file and publishes it on the Google play store for example. Sure an LLM now may still not be able to get everything perfect as far as it's outputs go. But surely there's already systems and workflows in place that will auto run your code, compile it, feed errors back to the LLM, some api to interact with cloud providers for hosting etc?
- dboreham 2y agoIt doesn't need to write tests: it can just use the application and figure out if it works.
- logicchains 2y agoThat's going to be much slower and more expensive than writing tests because image/video processing is slower and more expensive than writing tests. And because of lag in using the UI (and re-building the whole application from scratch after every change to test again).
- ClumsyPilot 2y agoBut it’s actually correct from a usability perspective
- drdeca 2y agoHm, what if instead of using video of the application… Ok, so if one can have one program snoop on all the rendering calls made by another program, maybe there could be a way of training a common representation of “an image of an application” and “the rendering calls that are made when producing a frame of the display for the application”? Hopefully in a way that would be significantly smaller than the full image data. If so, maybe rather than feeding in the video of the application, said representation could be applied to the rendering calls the application makes each frame, and this representation would be given as input as the model interacts with the application, rather than giving it the actual graphics? But maybe this idea wouldn’t work at all, idk. Like, I guess the rendering calls often involve image data in their arguments, and, you wouldn’t want to include the same images many time as the input to the encoding thing, as that would probably (or, I imagine) make it slower than just using the overall image of the application. I guess the calls are probably more pointing to the images in memory though, not putting an entire image on the stack. I don’t know enough about low-level graphics programming to know if this idea of mine makes any sense.
- mohsen1 2y agoThe idea that a lot of compute is moving towards inference has a huge consequence for the current "AI investments". This is bad news for NVDA particularly. The inference focused solutions have better economics than paying NVDA those huge margins (e.g. Grog)
- talldayo 2y agoNvidia can actually charge larger margins if inference compute goes down. It would enable them to manufacture more units of smaller GPUs using inferior and cheaper silicon, all of which would increase the profits per unit sold as well as the number of units they can manufacture. The industry has to find a way to separate itself from Nvidia's GPGPU technology if they want to stop being gouged. The issue is that nobody, not Apple, not AMD, not Intel, has been treating Nvidia's hardware as a serious threat.
- logicchains 2y ago>The issue is that nobody, not Apple, not AMD, not Intel, has been treating Nvidia's hardware as a serious threat Google has and they've built a much more cost efficient (for them) system: the TPU. They even rent them out, and in terms of cost per unit compute TPUs are significantly cheaper than renting GPUs from the big cloud providers. Amazon's also tried to do something similar with Trainium chips, however they're usefulness is more limited due to software issues (Amazon's much weaker at compiler development than Google, so Trainium software is quite slow and buggy).
- vidarh 2y agoFor inference Nvidia has more significant competition than for training. See Groq, Google's TPU's etc.
- pants2 2y agoPeople talk about Groq and Cerberus as competitors but it seems to me their manufacturing process makes the availability of those chips extremely limited. You can call up Nvidia and order $10B worth of GPUs and have them delivered the next week. Can't say the same for these specialty competitors.
- rybosome 2y agoFascinating. R1 really punches above its weight with respect to cost-per-token. As the article alluded to at the end, my thoughts immediately go to using R1 as a data generator for complex problems, since we have many examples of successful distillation into smaller models on well-defined tasks.
- mikejulietbravo 2y agoMike from Baseten here We're super proud to support this work. If you're thinking of running deepseek in production, give us a shout!
- fxttr 2y agoWe currently evaluate DeepSeek-R1 for our production system. We aren't done yet, but I think it's a match.
- mikejulietbravo 2y agoAwesome - we'd love to have our CEO/CTO chat with you and your team if you're interested. Shoot me a note at mike.bilodeau @ baseten.co and I'll make it happen!
- littlestymaar 2y agoEarlier today I read a reddit comment[1] about a guy who tried running the quantized version from unsloth[2] on 4xH100 and the results was underwhelming (it ended up costing $137 per 1 million tokens). Any idea of what they're doing wrong? [1]: https://www.reddit.com/r/LocalLLaMA/comments/1icphqa/how_to_run_deepseek_r1_on_4xh100/ https://www.reddit.com/r/LocalLLaMA/comments/1icphqa/how_to_... [2]: https://unsloth.ai/blog/deepseekr1-dynamic https://unsloth.ai/blog/deepseekr1-dynamic
- philipkiely 2y agoThey're using Llama.cpp which is an amazing tool for local inference but doesn't match fast inference frameworks like TensorRT-LLM/SGLang for production speeds and throughputs on Hopper GPUs. The Unsloth quantizations are really cool, but if you want to experiment with the R1 models in a smaller form factor the R1 Distills like Llama 70B are great and should run a lot faster as they take advantage of existing optimizations around inferencing llama-architecture models.
- littlestymaar 2y ago
- Stevvo 2y ago"The o3 system demonstrates the first practical, general implementation of a computer adapting to novel unseen problems" Yet, they said when it was announced: "OpenAI shared they trained the o3 we tested on 75% of the Public Training set. They have not shared more details. We have not yet tested the ARC-untrained model to understand how much of the performance is due to ARC-AGI data." These two statements are completely opposed. I can't take seriously anything this article says about o3.
- usaar333 2y agoNo they aren't. Every arc problem is novel - that's why it resisted deep learning for so long (and still does to a degree). We just don't know how much the model seeing what an arc problem is on the first place boosts its ability to solve them - that limited statement is all the author is making.
- joe_the_user 2y agoThe ARC prize was created last year. Arc hasn't resisted AI for very long. See; https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet
- usaar333 2y agoHuh? It was made in 2019
- joe_the_user 2y agoOK, I was following Wikipedia: In 2024, Chollet launched ARC Prize, a US$1 million competition to solve the ARC-AGI benchmark. I guess the ARC benchmark appeared in 2019 (https://arcprize.org/ https://arcprize.org/). So (shrug)
- 7thpower 2y agoThey are testing with a different dataset. The authors saying that they have not tested on the version of o3 that has not seen the training set.
- spyckie2 2y ago> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers are footing the bill to create new high quality data … which improves the model … which becomes better and more preferred by users … you get the idea. While I think this is an interesting hypothesis, I'm skeptical. You might be lowering the cost of your training corpus by a few million dollars, but I highly doubt you are getting novel, high quality data. We are currently in a world where SOTA base model seems to be capped at around GPT4o levels. I have no doubt that in 2-3 years our base models will compete with o1 or even o3... just it remains to be seen what innovations/optimizations get us there. The most promising idea is to use reasoning models to generate data, and then train our non-reasoning models with the reasoning-embedded data. But... it remains to be seen how much of the chain of thought reasoning you can really capture into model weights. I'm guessing some, but I wonder if there is a cap to multi-head attention architecture. If reasoning can be transferred from reasoning models to base models, OpenAI should have already trained a new model with o3 training data, right? Another thought is maybe we don't need to improve our base models much. It's sufficient to have them be generalists, and to improve reasoning models (lowering price, improving quality) going forward.
- mohsen1 2y agoevery time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now
- jeffbee 2y agoIf such labels are collected and used to retrain the model then yes. But these models are not learning online.
- jvanderbot 2y ago
- dr_dshiv 2y ago> The R1-Zero training process is capable of creating its own internal domain specific language (“DSL”) in token space via RL optimization. Um, what’s that now? Really?
- svdr 2y agoYeah, if I understand correctly AI will create it's own internal reasoning language through RL. In R1-Zero it was already a strange mix of languages. They corrected that for R1 to make the thinking useful for humans.
- dutchbookmaker 2y agoNot trying to be ironic but it would be interesting to see what this below would look like in the strange mix form: "If the model's actions involve generating tokens (like in language models), then optimizing these token outputs to maximize reward could lead the model to develop a consistent, efficient way of using tokens that's specific to the problem domain. This might look like a DSL because the tokens are used in a structured, perhaps abbreviated or symbolic way that's efficient for the task, not necessarily human-readable but effective for the model's internal processing."
- nlpnerd 2y agoThat is a slight exaggeration, extrapolation on the author's part. What happened was that RL training led to some emergent behavior in R1-Zero (chain-of-thought, and reflection) without being prompted or trained for explicitly. Don't see what is so domain specific about that though.
- sleepbyte 2y ago[dead]
- Developerx 2y ago[flagged]
- Developerx 2y ago[flagged]
- hnburnsy 2y ago>There are two major shifts happening in AI, economically speaking: You can now spend more $ to get higher accuracy and reliability Training $ is moving to inference $ >Both are going to drive a massive amount of demand for inference and neither will curtail the demand for more compute. In fact, they will increase the demand for compute. Is this Nvidia compute or something else?
- gorbypark 2y agoNvidia has much less of a moat on the inference side of things. Of course they still dominate the market right now for inference (in datacenters), but it's much easier for companies to move onto AMD or other solutions like Groq or whatever compared to trying to use non-Nvidia for training.
- cbracketdash 2y agoIt's a bit deceptive that o3 conveniently had access to ARC-prize-specific training material while r1 probably didn't. [0] [0] https://news.ycombinator.com/item?id=42763231 https://news.ycombinator.com/item?id=42763231
- usaar333 2y agoOverall good post, but feels like he has an axe to grind with LLMs to the point it is misleading: > Last week, DeepSeek published their new R1-Zero and R1 “reasoner” systems that is competitive with OpenAI’s o1 system on ARC-AGI-1. R1-Zero, R1, and o1 (low compute) all score around 15-20% – in contrast to GPT-4o’s 5%, the pinnacle of years of pure LLM scaling R1-zero gets 14% on private set which is the exact same score June Sonnet got; Sonnet, not 4o, is the pinnacle of pure LLM scaling
- octacat 2y agoI think deepseek accidentally also killed google for me, not just chatgpt. Because of the visible reasoning part.
- nogridbag 2y agoFrom what I read elsewhere (random reddit comment), the visible reasoning is just "for show" and isn't the process deepseek used to arrive at the result. But if the reasoning has value, I guess it doesn't matter even if it's fake.
- ziaowang 2y agoCan you provide a link to the comment? R1's technical report (https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...) says the prompt used for training is "<think> reasoning process here </think> <answer> answer here </answer>. User: prompt. Assistant:" This prompt format strongly suggests that the text between <think> is made the "reasoning" and the text between <answer> is made the "answer" in the web app and API (https://api-docs.deepseek.com/guides/reasoning_model https://api-docs.deepseek.com/guides/reasoning_model). I see no reason why deepseek should not do it this way, if not considering post-generation filtering. Plus, if you read table 3 of the R1 technical report, which contains an example of R1's chain of thought, its style (going back to re-evaluating the problem) resembles what I actually got in the COT in the web app.
- octacat 2y agoBad reddit comment though, try pair programming with it. Reasoning usually comments on your request, extends it, figures out which solution is the best and usable, backtracks if finds issues implementing it, proposes a new solution and verifies that it kinda makes sense. The result after that could actually look different though for usual questions (i.e. summarised in a way chatgpt answers on questions would look like). But it is usually very coherent with the code part, so if for example it has to choose from two libraries - it will use the one from the reasoning part, of course.
- cootsnuck 2y ago
- eslammhmad18 2y agoAnas
- jojobas 2y agoFunnily they didn't exclude anything forbidden from the training dataset. It will gladly tell you about the Tienanmen Massacre and what not.
- visarga 2y ago> R1-Zero removes the human bottleneck I disagree. It only removes the bottleneck to collecting math and code reasoning chains, not in general. The general case requires physical testing not just calculations, otherwise scientists would not need experimental labs. Discovery comes from searching the real world, it's where interesting things happen. The best interface between AI and the world are still humans, the code and math domains are just lucky to work without real world interaction.
- janalsncm 2y agoIn the case of ARC they are referring to verifiable math and reasoning problems. They still used SFT and model-based rewards for other domains.
- Bjorkbat 2y agoI'm still skeptical on the notion that we can remove the human bottleneck on code because code has verifiable solutions. It's true only to the extent that there's sufficient test coverage to prevent any unwanted side effects. Easy to do with straight forward problems, far more difficult with more complex as well as open-ended problems.
- mtrovo 2y agoThe fact that both systems scored well on ARC AGI 1 shows they can handle unseen challenges without heavy human input, unless I'm missing something about why you see humans as the best interface for real world exploration.
- georgiivanov 2y agoLeaving r1 at 15.8% percent with $0.06/task without scaling to medium and high compute is a bit misleading imo. The whole point of DeepSeek is the efficiency.
- Mr_Bees69 2y agoDoes anyone have a copy of those aforementioned garbled chains of thought?
- dakshgupta 2y ago>Generate chains-of-thought (CoT) for a problem domain. >Label the intermediary CoT steps using a combination of human experts (“supervised fine tuning” or SFT) and automated machines (“reinforcement learning” or RL). >Train base model using (2). This is remarkably intuitive and elegant. Seems analogous to the idea that humans can come up with new knowledge by synthesizing from their current knowledge. Theoretical sciences or creative arts for example.
- anna41517 2y ago[flagged]
- epictwow 2y agoHamlet says "to be or not to be", implying it's simply a choice. I choose to be!
- epictwow2 2y agoCommit theft at sea. Now, being a pirate, I'm confident in my "Arrr"s!