4 ms·
Is this why my memory consumption explodes when I increase the batch size of my inputs? Sorry I’m a relative GPU noob, but I find it fascinating to observe some
by stingraycharles 6y ago
Is this why my memory consumption explodes when I increase the batch size of my inputs? Sorry I’m a relative GPU noob, but I find it fascinating to observe some higher level interactions with eg pytorch; it seems like the GPU is a very leaky distraction, and you quickly find yourself battling GPU memory management, truncating input sizes and whatnot in order to deal with the insane memory requirements.
- banachtarski 6y agoBatch size is an orthogonal concept. The reason your memory increases so much is because with larger batches, you need to store more intermediate results before running your optimizer. This would be true with a pure CPU implementation of an SGD (stochastic gradient descent) optimizer as well.