4 ms·

For what it's worth, the article hits the nail on the head with my observations as well re: the model no longer being able to reason and go through the step by

For what it's worth, the article hits the nail on the head with my observations as well re: the model no longer being able to reason and go through the step by step processes mentioned. I have asked it to do several really simple tasks recently (e.g. list all headers in a link with exact parameters on how they are formatted), which it used to be able to do with ease, had it repeat back to me all instructions and parameters in order then it would completely ignore those instructions and do random things. I am using it less and less because of this. Maybe I just need agents?

KingOfCoders 3y ago
I have a lot of problems with getting the title of an URL. It worked, no longer does.
- capableweb 3y ago
  > I have a lot of problems with getting the title of an URL. It worked, no longer does. What do you mean exactly? Would be fun to try to replicate. You give ChatGPT a URL and expect it to be able to get the <title> tag from the HTML? Or you're pasting the whole HTML document?
capableweb 3y ago
I personally haven't experienced the same, ChatGPT and GPT4 via API seems mostly the same as before. Here is an example system prompt I'm using for some programming tasks: > Repeat what I've set as the requirements in other words to ensure you understand it. Describe concisely at least two different approaches you could take to solve the problem while explaining the reasoning for why it's a useful approach. Chose the best approach, then think through how the problem could be solved step-by-step. Finally implement each step and provide a full solution. Yesterday I used that to build (iteratively) a Rust CLI that downloads the last X HN comments have group-counts them by domain. Ended up being ~100 lines that GPT wrote by itself without any errors along the way.
DonaldPShimoda 3y ago
> the model no longer being able to reason The model was never able to reason; it was only able to generate responses that masqueraded as reasoning. Thorough investigations of its "reasoning process" reveal this at every stage of its development.
- whatshisface 3y ago
  Okay, then they're saying it's no longer able to masquerade as reasoning.
  - DonaldPShimoda 3y ago
    I can appreciate that, but in the context of LLMs where people are frequently drumming up discourse based on faulty beliefs about the systems' capabilities, I think it is important to be clear.

KingOfCoders 3y ago

I have a lot of problems with getting the title of an URL. It worked, no longer does.

capableweb 3y ago
> I have a lot of problems with getting the title of an URL. It worked, no longer does. What do you mean exactly? Would be fun to try to replicate. You give ChatGPT a URL and expect it to be able to get the <title> tag from the HTML? Or you're pasting the whole HTML document?

capableweb 3y ago

I personally haven't experienced the same, ChatGPT and GPT4 via API seems mostly the same as before. Here is an example system prompt I'm using for some programming tasks: > Repeat what I've set as the requirements in other words to ensure you understand it. Describe concisely at least two different approaches you could take to solve the problem while explaining the reasoning for why it's a useful approach. Chose the best approach, then think through how the problem could be solved step-by-step. Finally implement each step and provide a full solution. Yesterday I used that to build (iteratively) a Rust CLI that downloads the last X HN comments have group-counts them by domain. Ended up being ~100 lines that GPT wrote by itself without any errors along the way.

DonaldPShimoda 3y ago

> the model no longer being able to reason The model was never able to reason; it was only able to generate responses that masqueraded as reasoning. Thorough investigations of its "reasoning process" reveal this at every stage of its development.

whatshisface 3y ago
Okay, then they're saying it's no longer able to masquerade as reasoning.
- DonaldPShimoda 3y ago
  I can appreciate that, but in the context of LLMs where people are frequently drumming up discourse based on faulty beliefs about the systems' capabilities, I think it is important to be clear.