3 ms·
Similar thing sometimes happens in deep learning field. Just a couple of weeks ago I helped a coworker reproduce a result from a paper. He had been debugging it
by p1esk 6y ago
Similar thing sometimes happens in deep learning field. Just a couple of weeks ago I helped a coworker reproduce a result from a paper. He had been debugging it for several days. Took me about 5 minutes to make a correct guess that his batch size is too small for batchnorm to be effective. He trained the model on 8 GPUs, using a decent batch size, but he didn't realize that the default batchnorm op in Pytorch isn't synchronizing batch statistics across GPUs, so the effective batch size was in fact 8x smaller. How was I able to see it so quickly? Experience.
- michaelhoffman 6y agoThis is what Raymond Chen calls the "psychic debugger".
- taneq 6y agoAlso sometimes called 'spidey sense'. You don't know how you know, but your subconscious synthesizes the idea somehow.
- robocat 6y agoI think this type of non-rational intuition shows a combination of intelligence and experience. “It was obvious” gets said by a smart person for things that are opaque to someone that lacks a correctly functioning intuition.
- taneq 6y agoI'd say it's a product of unconscious competence. "I can't articulate why, but this bit here is the problem." I do maintain, though, that if you can't explain your reasoning for something then you don't understand it. Doesn't mean you're wrong, though!