3 ms·

Is COT considered a deep area of engineering/research or is it considered as simply a temporary hack that will be obsolete in the near or medium term?

Ozzie_osman 4y ago
Despite the fancy name (Chain Of Thought) and multiple arxiv papers, I'd definitely consider it a temporary hack that the models will overcome as they get more powerful and as they are better-trained to avoid hallucination. From this paper, it looks like ChatGPT may have already overcome it for some types of questions. What may be here to stay might be more task-specific prompting to break a problem down. For example, a variation might be "Let's think step by step, making sure we do or consider X before we do Y". By way of analogy, when my toddler was really young, if he couldn't figure something out, I'd ask him to break it down into steps (very similar to general Cot prompting). Now, he rarely needs that, but still needs more occasional, task-specific "prompting", eg "Let's think about the weather, and what we're going to be doing, before deciding what to wear".
- JieJie 4y ago
  Isn't breaking things down into steps always going to be a part of solving any problem? I think of how I was told to approach problems while developing software in this manner. Maybe the LLM gets so smart that it doesn't need to do this, but that I would definitely want to call a super-intelligence, because I sure can't solve big problems without breaking them down into smaller parts. But maybe you are on to something when you say there are some things that we've just done so much that we don't really have to think them through to accomplish them anymore. I don't know what that would say about an AI that was smart enough it could just spit out the right answer to literally any question we could devise. That would be quite a feat of engineering.
  - vineyardmike 4y ago
    > But maybe you are on to something when you say there are some things that we've just done so much that we don't really have to think them through to accomplish them anymore. It’s like multiplication tables for thought. Using the “what to wear” example, you’ve essentially already just memorized what to wear for every occasion so you rarely need to think about it. Eg you know what is work appropriate, and if it’s raining or snowing or hot you’re still fine. But a wedding for a foreign culture you’re not familiar with in a city with weather you’re not familiar with, and you might not immediately know what to wear, so you have to break it down.
    - JieJie 4y ago
      I've taken to calling these things "language golems" because they seem to be able to animate language itself and unlock all these hidden qualities that we seem to know are there, take for granted, and never seemed to ask ourselves how they worked on a fundamental level. Even if they are "stochastic parrots" that are no more effective at elucidating truth than Ouija boards, they are still better than the autocorrect on my phone right now.
deleted 4y ago
[deleted]
travisjungroth 4y ago
I don’t know of a consensus for it to be “considered” one way or another. I think explicitly asking for COT is temporary, and it will move to a layer that’s encapsulated from most users. It will persist there as long as long as the language layer is doing the “thinking”. If the language layer is completely offloading that and just translating, it won’t be needed. I’ve been explaining it to people as “if ChatGPT can think, it can only think out loud”. Because of the way LLMs were trained, nudging it to do this will continue to be valuable. Most writing about anything difficult is product, not process. Articles get drafts before being published. People think about answers before writing them down. How to Solve It does a great job explaining this about math problems. The steps to the proof are not the steps to creating the proof. So when you go to solve a problem by mimicking the solutions to problems, something is missing. People will either need to train LLMs to do it, or continue to ask for it in prompts.

Ozzie_osman 4y ago

Despite the fancy name (Chain Of Thought) and multiple arxiv papers, I'd definitely consider it a temporary hack that the models will overcome as they get more powerful and as they are better-trained to avoid hallucination. From this paper, it looks like ChatGPT may have already overcome it for some types of questions. What may be here to stay might be more task-specific prompting to break a problem down. For example, a variation might be "Let's think step by step, making sure we do or consider X before we do Y". By way of analogy, when my toddler was really young, if he couldn't figure something out, I'd ask him to break it down into steps (very similar to general Cot prompting). Now, he rarely needs that, but still needs more occasional, task-specific "prompting", eg "Let's think about the weather, and what we're going to be doing, before deciding what to wear".

JieJie 4y ago
Isn't breaking things down into steps always going to be a part of solving any problem? I think of how I was told to approach problems while developing software in this manner. Maybe the LLM gets so smart that it doesn't need to do this, but that I would definitely want to call a super-intelligence, because I sure can't solve big problems without breaking them down into smaller parts. But maybe you are on to something when you say there are some things that we've just done so much that we don't really have to think them through to accomplish them anymore. I don't know what that would say about an AI that was smart enough it could just spit out the right answer to literally any question we could devise. That would be quite a feat of engineering.
- vineyardmike 4y ago
  > But maybe you are on to something when you say there are some things that we've just done so much that we don't really have to think them through to accomplish them anymore. It’s like multiplication tables for thought. Using the “what to wear” example, you’ve essentially already just memorized what to wear for every occasion so you rarely need to think about it. Eg you know what is work appropriate, and if it’s raining or snowing or hot you’re still fine. But a wedding for a foreign culture you’re not familiar with in a city with weather you’re not familiar with, and you might not immediately know what to wear, so you have to break it down.
  - JieJie 4y ago
    I've taken to calling these things "language golems" because they seem to be able to animate language itself and unlock all these hidden qualities that we seem to know are there, take for granted, and never seemed to ask ourselves how they worked on a fundamental level. Even if they are "stochastic parrots" that are no more effective at elucidating truth than Ouija boards, they are still better than the autocorrect on my phone right now.

deleted 4y ago

[deleted]

travisjungroth 4y ago

I don’t know of a consensus for it to be “considered” one way or another. I think explicitly asking for COT is temporary, and it will move to a layer that’s encapsulated from most users. It will persist there as long as long as the language layer is doing the “thinking”. If the language layer is completely offloading that and just translating, it won’t be needed. I’ve been explaining it to people as “if ChatGPT can think, it can only think out loud”. Because of the way LLMs were trained, nudging it to do this will continue to be valuable. Most writing about anything difficult is product, not process. Articles get drafts before being published. People think about answers before writing them down. How to Solve It does a great job explaining this about math problems. The steps to the proof are not the steps to creating the proof. So when you go to solve a problem by mimicking the solutions to problems, something is missing. People will either need to train LLMs to do it, or continue to ask for it in prompts.