3 ms·
I have one more - LLMs are terrible at counting and arithmetic - if your code gen relies on cutting off the first two words of a constant string - you better ch
by torginus 2y ago
I have one more - LLMs are terrible at counting and arithmetic - if your code gen relies on cutting off the first two words of a constant string - you better check if you need to cut off 12 characters like the LLM says. If it adds 2 numbers, it might be suspect. If you need it to decode a byte sequence, where getting the numbers from the exact right position is necessary.. you get the idea.
Took me a day to debug my LLM-generated code - and of course, like all fruitless and long debugging sessions, this one started with me assuming that it can't possibly get this wrong - yet it did.