2 ms·
There are ChatGPT resistant questions you can ask. ChatGPT recognizes the question but doesn't actually think about it, so if you give it the river crossing pro
by fragmede 3y ago
There are ChatGPT resistant questions you can ask. ChatGPT recognizes the question but doesn't actually think about it, so if you give it the river crossing problem (farmer, fox, sheep, and grain need to cross a river) but tell it the boat can take all the items, it won't actually read those details and blithely solve the problem the expected way. Give candidates a problem that's trivially solvable if you actually read the question and see if they try and solve it the ChatGPT way.
- hirsin 3y agoIndeed - with 3.5 at least it does fail the easy mode riddle https://chat.openai.com/share/b3761807-551d-4cfc-b291-6d37ee2de4dd https://chat.openai.com/share/b3761807-551d-4cfc-b291-6d37ee...
- fragmede 3y agoit's a fun problem to explore, and gpt-4 doesn't do any better. swapping in other things doesn't help because it internally recognizes it as the river crossing problem and proceeds to solve it normally. I was able to get it to two shot it with a lot of coaching but yeah, it's a trip.
- ric2b 3y agoThe downside is that you're now wasting interview time on the river crossing problem instead of actually relevant questions for the job you're hiring for.
- fragmede 3y agoDon't literally use the river crossing problem, but it's existence implies there is a form of question that ChatGPT will solve for where someone actually reading the prompt can solve trivially but someone using ChatGPT will get stuck on. You're already asking Leetcode questions that are irrelevant for the job you're hiring for. What's the problem with asking one more to test for cheaters?