3 ms·

I have a different view. I have been trying to use GPT for some tasks like NP reductions, Linear Programming formulations, Graph problems, etc. and the results

I have a different view. I have been trying to use GPT for some tasks like NP reductions, Linear Programming formulations, Graph problems, etc. and the results are maddeningly bad. Seriously, I have raged in front of my computer like I hadn't raged since the days of the NES Ghostbusters game. The amount of mistakes is incredible and the model seems to constantly feed into vicious circles of "I am sorry, you are correct, I made a mistake" then moving a few things around then being corrected again and ending up on the original mistake. Two interesting things it did for me lately: insisted that 8 + 6 = 11 and also proved that P = NP. I don't know which solutions it may be providing you, but it can't be anything too complex, or at least not too abstract. edit: typo

eugmill 3y ago
I feel like every chatgpt thread needs every comment to be prepended with 3.5 or 4. It's incredibly difficult to tell when someone is having a bad time with 3.5, which has many known limitations, or 4 which tends to be more better at problem solving (though still makes dumb mistakes). I think the "GPT is amazing" vs "GPT is useless" debate is just going to get more confusing as more versions are released.
billythemaniam 3y ago
I'm not surprised. LLMs are not good at those problems. There is a lot of hype, it not good at every problem, but it is quite good at some of them. If you have a classical NLP task, then it is good, particularly GPT4. If you have a generative problem where you don't care about mistakes, such as grocery list, marketing copy rough draft, etc, then it is good. LLMs are not good at search, math, encyclopedias, logic engines. Maybe some day they will be, but not yet.
lacker 3y ago
ChatGPT (specifically GPT4) is very good at complicated but shallow questions. For example, "I'm setting up a new TypeScript + Svelte app, I made some changes to the config that I thought were good, but when I try to run the dev server, I get this error." And then paste in your incomprehensible ten line error. Any time you are using some software tool that you're not very familiar with, and you get an error message that you don't understand, try asking GPT4 to explain it to you. It isn't so great at deep, theoretical, algorithmic questions. "NP reductions" are probably not a great fit. Think of it as a research assistant that has a broad understanding of every technology in the world, but isn't as smart as you are about your specific area of expertise.

eugmill 3y ago

I feel like every chatgpt thread needs every comment to be prepended with 3.5 or 4. It's incredibly difficult to tell when someone is having a bad time with 3.5, which has many known limitations, or 4 which tends to be more better at problem solving (though still makes dumb mistakes). I think the "GPT is amazing" vs "GPT is useless" debate is just going to get more confusing as more versions are released.

billythemaniam 3y ago

I'm not surprised. LLMs are not good at those problems. There is a lot of hype, it not good at every problem, but it is quite good at some of them. If you have a classical NLP task, then it is good, particularly GPT4. If you have a generative problem where you don't care about mistakes, such as grocery list, marketing copy rough draft, etc, then it is good. LLMs are not good at search, math, encyclopedias, logic engines. Maybe some day they will be, but not yet.

lacker 3y ago

ChatGPT (specifically GPT4) is very good at complicated but shallow questions. For example, "I'm setting up a new TypeScript + Svelte app, I made some changes to the config that I thought were good, but when I try to run the dev server, I get this error." And then paste in your incomprehensible ten line error. Any time you are using some software tool that you're not very familiar with, and you get an error message that you don't understand, try asking GPT4 to explain it to you. It isn't so great at deep, theoretical, algorithmic questions. "NP reductions" are probably not a great fit. Think of it as a research assistant that has a broad understanding of every technology in the world, but isn't as smart as you are about your specific area of expertise.