6 ms·
I'm not a huge fan of using these kind of riddles or gotchas. Other comments have riddle variants which feel also like ways of tripping someone up - if you don'
by IanCal 2y ago
I'm not a huge fan of using these kind of riddles or gotchas. Other comments have riddle variants which feel also like ways of tripping someone up - if you don't spot the change you fail. And what's more the originals are things that lots of people struggle with (that's why they're riddles not basic questions).
There's also little prompting, which feels like calling up a random person and demanding they solve a riddle straight away without talking it through.
Part of the assumption here is that if llms don't write the right code they don't understand what people know. I'd wager that a huge number of people couldn't solve this puzzle yet fully understand what different people have their own internal thoughts and knowledge.
- rtlacn 2y agoHumans can solve these riddles even if they have never seen a similar one. LLMs need to have seen at least a similar riddle with the same structure and/or code to plagiarize. Humans can deal with modified riddles if they are alert and awake. An LLM is always alert and awake, so it should be able to handle modified riddles, but it cannot.
- mirekrusin 2y agoNot to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.
- samrus 2y agoYeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit
- Terretta 2y ago> An LLM can't be excused... LLMs are always operating at their best Depends how much you turn down the heat.
- mirekrusin 2y agoYes, but they will only keep getting better; whether we like it or not sooner or later we’re going to be hiring api keys, not people.
- k__ 2y ago"they will only keep getting better" I don't think that's a given. It could very well be that they only get smaller, faster, or even worse when they accidentally start training them with hallucinations that spread on the web.
- aithrowawaycomm 2y agoThe more dangerous - and far more ridiculous - assumption is that if the LLM does write the correct code then it does understand what people know. I am amazed at how common that view is here.
- ifdefdebug 2y ago> if you don't spot the change you fail But that's exactly how real world works too.
- Fripplebubby 2y agoThis has not been my experience with the real world. Riddles and gotchas have played a very small role, so far.
- shermantanktop 2y agoUnfortunately they do show up in tech interviews. “What’s the correct way to delete an element from a Java array while iterating over it?” “Well I suppose you’d need to avoid invalidating the iterator state but I don’t recall the…” “BZZT!”
- johnisgood 2y agoI am not sure I understand. It seems very easy. You cannot directly remove an element from an array, you could create a new array that excludes that element. Arrays have a fixed size once declared, and I cannot imagine anyone who has written some code not knowing. :/
- HappMacDonald 2y agoI "have written some code" but it's been decades since I've done anything significant in Java in particular, and every language handles arrays (and/or data structures that get called "arrays") differently.
- johnisgood 2y agoThe terminology may be confusing, yes, although you would rather call them dynamic arrays or lists (like in Common Lisp). Plus you did say "decades", that is a long time nevertheless. I was not referring to people who have written some code decades ago, of course.