4 ms·

A lot of the niceness about DeepSeek-R1's usage in coding is that you can see the thought process, which (IME) has been more useful than the final answer. It m

by m11a 2y ago

A lot of the niceness about DeepSeek-R1's usage in coding is that you can see the thought process, which (IME) has been more useful than the final answer. It may well be that o1's chain of thought reasoning trace is also quite good. But they hide it as a trade secret and supposedly ban users for trying to access it, so it's hard to know.

m11a 2y ago
One example from today: I had a coding bug which I asked R1 about. The final answer wasn't correct, but adapting an idea from the CoT trace helped me fix the bug. o1's answer was also incorrect. Interestingly though, R1 struggled in part because it needed the value of some parameters I didn't provide, and instead it made an incorrect assumption about its value. This was apparent in the CoT trace, but the model didn't mention this in its final answer. If I wasn't able to see the trace, I'd not know what was lacking in my prompt, and how to make the model do better. I presume OpenAI kept their traces a secret to prevent their competitors from training models with it, but IMO they strategically err'd in doing so. If o1's traces were public, I think the hype around DS-R1 would be relatively less (and maybe more limited to the lower training costs and the MIT license, and not so much its performance and usefulness.)
- manmal 2y ago
  Do you use Continue.dev or similar tools to load code into the context, or do you copypaste into their web chat?
- gwd 2y ago
  > I presume OpenAI kept their traces a secret to prevent their competitors from training models with it At some point there was a paper they'd written about it, and IIRC the logic presented was like this: - We (the OpenAI safety people) want to be able to have insight into what o1 is actually thinking, not a self-censored "people are watching me" version of its thinking. - o1 knows all kinds of potentially harmful information, like how to make bombs, how to cook meth, how to manipulate someone, etc, which could "cause harm" if seen by an end-user So the options as they saw it were: 1. RLHF both the internal thinking and the final output. In this case the thought process would avoid saying things that might "cause harm", and so could be shown to the user. But they would have a less clear picture of what the LLM was "actually" thinking, and the potential state space of exploration would be limited due to the self-censorship. 2. Only RLHF the final output. In this case, they can have a clearer picture into what the LLM is "actually" thinking (and the LLM could potentially explore the state space more fully without risking about causing harm), but thought process could internally mention things which they don't want the user to see. OpenAI went with #2. Not sure what DeepSeek has done -- whether they have RLHF'd the CoT as well, or just not worried as much about it.
fibers 2y ago
how many reported cases of banning are there? that sounds insane for asking it to print out its chain of thought
d3nj4l 2y ago
I have a lot of fun just posting a function into R1, saying "Improve this" and reading the chain of thought. Lots of insight in there that I would usually miss or glance over.
satvikpendem 2y ago
This is generally how I use LLMs anyway, as brainstorming tools, rather than writing code.
emporas 2y ago
I tried a month back o1 and Qwen with chain of thought QwQ, to explain to me some chemical reactions, QwQ got it correct, and o1 got it wrong. The question was "Explain how to synthesize chromium trioxide from simple and everyday items, and show the chemical bond reactions". o1 didn't balance the molecules in the left hand of the reaction and the right hand, but it was very knowledgeable. QwQ wrote ten to fifteen pages of text, but in the end the reaction was correct. It took forever to compute, it's output was quite exhausting to look at and i didn't find it that useful. Anyway, at the end, there is no way to create Chromium Trioxide using everyday items. I thought maybe i could mix some toothpaste and soap and get it.

by m11a 2y ago

m11a 2y ago

One example from today: I had a coding bug which I asked R1 about. The final answer wasn't correct, but adapting an idea from the CoT trace helped me fix the bug. o1's answer was also incorrect. Interestingly though, R1 struggled in part because it needed the value of some parameters I didn't provide, and instead it made an incorrect assumption about its value. This was apparent in the CoT trace, but the model didn't mention this in its final answer. If I wasn't able to see the trace, I'd not know what was lacking in my prompt, and how to make the model do better. I presume OpenAI kept their traces a secret to prevent their competitors from training models with it, but IMO they strategically err'd in doing so. If o1's traces were public, I think the hype around DS-R1 would be relatively less (and maybe more limited to the lower training costs and the MIT license, and not so much its performance and usefulness.)

manmal 2y ago
Do you use Continue.dev or similar tools to load code into the context, or do you copypaste into their web chat?
gwd 2y ago
> I presume OpenAI kept their traces a secret to prevent their competitors from training models with it At some point there was a paper they'd written about it, and IIRC the logic presented was like this: - We (the OpenAI safety people) want to be able to have insight into what o1 is actually thinking, not a self-censored "people are watching me" version of its thinking. - o1 knows all kinds of potentially harmful information, like how to make bombs, how to cook meth, how to manipulate someone, etc, which could "cause harm" if seen by an end-user So the options as they saw it were: 1. RLHF both the internal thinking and the final output. In this case the thought process would avoid saying things that might "cause harm", and so could be shown to the user. But they would have a less clear picture of what the LLM was "actually" thinking, and the potential state space of exploration would be limited due to the self-censorship. 2. Only RLHF the final output. In this case, they can have a clearer picture into what the LLM is "actually" thinking (and the LLM could potentially explore the state space more fully without risking about causing harm), but thought process could internally mention things which they don't want the user to see. OpenAI went with #2. Not sure what DeepSeek has done -- whether they have RLHF'd the CoT as well, or just not worried as much about it.

fibers 2y ago

how many reported cases of banning are there? that sounds insane for asking it to print out its chain of thought

d3nj4l 2y ago

I have a lot of fun just posting a function into R1, saying "Improve this" and reading the chain of thought. Lots of insight in there that I would usually miss or glance over.

satvikpendem 2y ago

This is generally how I use LLMs anyway, as brainstorming tools, rather than writing code.

emporas 2y ago

I tried a month back o1 and Qwen with chain of thought QwQ, to explain to me some chemical reactions, QwQ got it correct, and o1 got it wrong. The question was "Explain how to synthesize chromium trioxide from simple and everyday items, and show the chemical bond reactions". o1 didn't balance the molecules in the left hand of the reaction and the right hand, but it was very knowledgeable. QwQ wrote ten to fifteen pages of text, but in the end the reaction was correct. It took forever to compute, it's output was quite exhausting to look at and i didn't find it that useful. Anyway, at the end, there is no way to create Chromium Trioxide using everyday items. I thought maybe i could mix some toothpaste and soap and get it.