3 ms·

> GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates. I'm a programmer and I've used GPT4

> GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates. I'm a programmer and I've used GPT4 for a variety of tasks (well, tried to). The results have been mediocre on average, usually syntactically correct, but more often than not, semantically incorrect. It usually ends in frustration as GPT-4 keeps responding confidently incorrect answers and upon the slightest expression of doubt, it tends to spin in circles. ChatGPT: <Implausible response #1> Me: Are you sure? Reasons A, B, C [...] ChatGPT: I apologize... <implausible response #2> Me: Are you sure? Reasons D, E, F [...] ChatGPT: I apologize ... <repeats implausible response #1> I'd like to know what people who are so impressed with GPT-4's programming capabilities are doing? It must be TODO apps, solving leetcode problems, writing basic Express.js routers, some basic React components and code for <one of the top 10 popular libraries>. The kind of things ChatGPT has seen a million of examples of.

kazinator 4y ago
To use the tool effectively, you can't use Socratic questioning on it, or not always. You have to already know how to code the problem, so that you can spot what is wrong, and, if necessary, tell the thing how to fix it. The fix is not always going to come from the training data; it needs to scrape it from what you give away in your follow-up question, together with its training data. I went through an exercise in which I encoded a paragraph from Edgar Allen Poe with a Vigenere cipher. I presented this to GPT4 and asked it to crack it. First I had to persuade it that it's ethical; it's a cipher I made myself and it's not my secret. We worked out a protocol by which it can ask me questions which show that I know the plaintext. It was a quite a long chat, during which I had to give away many hints. In the end I basically gave the answer away, and the thing acted like it cracked the key. Then I reproached it and admitted that yes, it didn't crack the key but used my hint to locate the E. A. Poe passage. Basically, if you know the right answer to something, and GPT4 isn't getting it, it will not get it until you drop enough hints. As you drop the hints, it will produce rhetoric suggesting that it's taking credit for the solution.

kazinator 4y ago

To use the tool effectively, you can't use Socratic questioning on it, or not always. You have to already know how to code the problem, so that you can spot what is wrong, and, if necessary, tell the thing how to fix it. The fix is not always going to come from the training data; it needs to scrape it from what you give away in your follow-up question, together with its training data. I went through an exercise in which I encoded a paragraph from Edgar Allen Poe with a Vigenere cipher. I presented this to GPT4 and asked it to crack it. First I had to persuade it that it's ethical; it's a cipher I made myself and it's not my secret. We worked out a protocol by which it can ask me questions which show that I know the plaintext. It was a quite a long chat, during which I had to give away many hints. In the end I basically gave the answer away, and the thing acted like it cracked the key. Then I reproached it and admitted that yes, it didn't crack the key but used my hint to locate the E. A. Poe passage. Basically, if you know the right answer to something, and GPT4 isn't getting it, it will not get it until you drop enough hints. As you drop the hints, it will produce rhetoric suggesting that it's taking credit for the solution.