4 ms·
byte alignment has always been a consideration for high performance computing. this alludes to a fascinating, yet elementary, fact about computer science to me
by thakoppno 4y ago
byte alignment has always been a consideration for high performance computing.
this alludes to a fascinating, yet elementary, fact about computer science to me: there’s a physical atomic constraint in every algorithm.
- dekhn 4y agothat's not byte alignment, though- those constraints are what can be held in GPU RAM during a training batch, which is subject to a number of limits, such as "optimal texture size is a power of 2 or the next power of 2 larger than your preferred size". Byte alignment would be more like "it's three channels of data, but we use 4 bytes (wasting 1 byte) to keep the data aligned on a platform that only allows word-level access"
- thakoppno 4y agothanks for the insight. you obviously understand the domain better than me. let me try and catch up before I say anything more.