3 ms·
Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Lee
by zamnos 4y ago
Is it? According to OpenAI's paper
GPT-4 GPT-4 (no vision) GPT-3.5
Leetcode (easy) 31 / 41 31 / 41 12 / 41
Leetcode (medium) 21 / 80 21 / 80 8 / 80
Leetcode (hard) 3 / 45 3 / 45 0 / 45
https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5.
So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the real problems that aren't just gluing some libraries together your job is safe.
- yieldcrv 4y agopeople are impressed because you never need to do leetcode hard problems in any business anywhere in your career to make a few billion dollars. I would fire anybody using company time trying to reinvent some academic search thing instead of just using the hashmap.
- zamnos 4y agoReimplementing quicksort in a language the rest of the company doesn't use in a part of the software that no one ever touches is fireable, sure, but I suspect that on the way to ChatGPT there were more than a couple "Leedcode hard"-grade problems that OpenAI engineers and PhDs solved because the library for that didn't exist before.
- gptgpp 4y ago[dead]
- dougmwne 4y agoThat’s better than a lot of professional engineers can do, but I agree that it is unlikely to get hired in a senior FAANG role.
- capableweb 4y agoWhat is "Leetcode (hard)" specifically? Searching for it brought me to a website (https://leetcode.com/ https://leetcode.com/) but I'm assuming that is referring to some special list of problems that are meant to be harder than others, but I cannot find that.
- zamnos 4y agoPick from this list: https://leetcode.com/problemset/all/?difficulty=HARD https://leetcode.com/problemset/all/?difficulty=HARD
- HDThoreaun 4y agoThe problems are all labeled with a difficulty.
- famouswaffles 4y agoMy dude, putting aside the fact that leetcode problem solving isn't factoring in the majority of programming use cases... you do realize this is better than a lot of professional programmers right ? It's truly amazing seeing the posts shift in real time.
- Jensson 4y agoAI will overperform at questions whose answers are in its training set. These leetcode questions has lots of answers posted online, any human with internet access can just google those and copy paste the solution and get a better score than these language models. This effect is stronger the larger the model, so likely most of the improvements here is that the model has better memory of the solutions that are already out there.
- TrackerFF 4y agoWhat this tells me is that doing technical interviews online is going to become even more of a hassle. Either you'll have to completely surrender yourself to a battery of various "anti-cheating" software, or the problem difficulty is going to go through the roof.