4 ms·

Performance is not the only point of consideration. There are probably features on the TX2 that are not found on the TX1, though I haven't actually looked at t

by k_sze 9y ago

Performance is not the only point of consideration. There are probably features on the TX2 that are not found on the TX1, though I haven't actually looked at the difference. For instance, IIRC, the hardware accelerated jpeg decoding is pretty limited on the TK1, compared to the TX1/TK2. Basically you can decode and display (m)jpeg really fast, but you have little control over the output pixel format. It's a black box hidden in nvidia's special hardware. If you try to programmatically get at the pixel data, you'll find that the pointer is actually DMA-mapped, which means it's extremely slow to memcpy from, which defeats the purpose of hw acceleration.

DiThi 9y ago
Another point of consideration is the memory. Sometimes a data set can't be easily split like workloads, if at all.
arghwhat 9y ago
I don't get your claim about DMA memory being slow to memcpy from. DMA memory isn't special, apart from being non-relocatable and contiguous within an allocation. Things would be much worse if the copy to main memory wasn't done with DMA.
- andars 9y ago
  Maybe he means that that particular allocation is write-combined/uncached?
  - arghwhat 9y ago
    Apart from the fact that the cache needs to be invalidated before/after a DMA transfer to ensure cache coherency (automatically or not, depends on platform), DMA allocations have no problem with caches.
    - andars 9y ago
      DMA allocations have no problems with caches, but in some situations you can get a performance boost by marking as write-combined if the CPU only writes to the memory once and then hands it off. This results in less cache being occupied with useless data and can be beneficial due to coalesced writes, but will result in horrifically slow CPU reads from the region. I am suggesting that the author conflated "DMA allocation" with "write-combined allocation".
- k_sze 9y ago
  The explanation I got (from somebody else a while back) is this: DMA is only fast if it's accessed in large blocks, but memcpy is not designed to work that way. The hw accelerated jpeg decoding on the Jetson TK1 is backed by something called NVMM. I suspect it's a small chunk of memory that is closer to the special hw decoder, not part of the main 2 GiB RAM. When you decode jpeg, you get a DMA-mapped pointer to it. When you memcpy from it, memcpy tries to copy the data one byte at a time (or in very small chunks, if vectorized). So what happens is that in order to copy data from a large block of DMA-mapped memory, memcpy makes many requests for single bytes or very small chunks, but DMA ends up fetching and staging the same large block multiple times for memcpy to read. The same problem applies to any other operation that reads from the DMA-mapped memory in very small chunks. There is also no way to make the DMA-mapped pixel data directly available to a CUDA context, which is what Jetson board users would probably care about. I'm not usually a system programmer so my explanation may be a bit fuzzy or inaccurate, but that's the gist of it.
  - arghwhat 9y ago
    DMA isn't something you access. Direct Memory Access means that an external device can issue reads and writes to main system memory (you call such a transaction a "DMA transfer"). What you access is entirely normal memory. A DMA transfer is done entirely by the external device, without any intervention by you or the CPU (apart from potentially a cache invalidation before/after, depending on platform). From the point of view of the CPU (and therefore you), data just magically appears. You don't need to memcpy anything anywhere, unless you need to make sure the external device doesn't accidentally overwrite things (who knows, it might be stupid). You can read and write directly to wherever it appeared as if it was any other memory. Caching still applies, although a fresh DMA transfer will of course be a cache miss. There is a little bit of magic related to memory regions used for DMA transfers, but that is invisible to the end-user. 'memcpy' does not discriminate, and unless you're copying a single byte, memcpy will never operate on byte-sized chunks. Source: I do driver development for devices that DMA at extremely high rates (we're desperately waiting for PCIe 4.0 to become normal—PCIe3.0x16's 126Gb/s is way too slow for us).

by k_sze 9y ago

DiThi 9y ago

Another point of consideration is the memory. Sometimes a data set can't be easily split like workloads, if at all.

arghwhat 9y ago

I don't get your claim about DMA memory being slow to memcpy from. DMA memory isn't special, apart from being non-relocatable and contiguous within an allocation. Things would be much worse if the copy to main memory wasn't done with DMA.

andars 9y ago
Maybe he means that that particular allocation is write-combined/uncached?
- arghwhat 9y ago
  Apart from the fact that the cache needs to be invalidated before/after a DMA transfer to ensure cache coherency (automatically or not, depends on platform), DMA allocations have no problem with caches.
  - andars 9y ago
    DMA allocations have no problems with caches, but in some situations you can get a performance boost by marking as write-combined if the CPU only writes to the memory once and then hands it off. This results in less cache being occupied with useless data and can be beneficial due to coalesced writes, but will result in horrifically slow CPU reads from the region. I am suggesting that the author conflated "DMA allocation" with "write-combined allocation".
k_sze 9y ago
The explanation I got (from somebody else a while back) is this: DMA is only fast if it's accessed in large blocks, but memcpy is not designed to work that way. The hw accelerated jpeg decoding on the Jetson TK1 is backed by something called NVMM. I suspect it's a small chunk of memory that is closer to the special hw decoder, not part of the main 2 GiB RAM. When you decode jpeg, you get a DMA-mapped pointer to it. When you memcpy from it, memcpy tries to copy the data one byte at a time (or in very small chunks, if vectorized). So what happens is that in order to copy data from a large block of DMA-mapped memory, memcpy makes many requests for single bytes or very small chunks, but DMA ends up fetching and staging the same large block multiple times for memcpy to read. The same problem applies to any other operation that reads from the DMA-mapped memory in very small chunks. There is also no way to make the DMA-mapped pixel data directly available to a CUDA context, which is what Jetson board users would probably care about. I'm not usually a system programmer so my explanation may be a bit fuzzy or inaccurate, but that's the gist of it.
- arghwhat 9y ago
  DMA isn't something you access. Direct Memory Access means that an external device can issue reads and writes to main system memory (you call such a transaction a "DMA transfer"). What you access is entirely normal memory. A DMA transfer is done entirely by the external device, without any intervention by you or the CPU (apart from potentially a cache invalidation before/after, depending on platform). From the point of view of the CPU (and therefore you), data just magically appears. You don't need to memcpy anything anywhere, unless you need to make sure the external device doesn't accidentally overwrite things (who knows, it might be stupid). You can read and write directly to wherever it appeared as if it was any other memory. Caching still applies, although a fresh DMA transfer will of course be a cache miss. There is a little bit of magic related to memory regions used for DMA transfers, but that is invisible to the end-user. 'memcpy' does not discriminate, and unless you're copying a single byte, memcpy will never operate on byte-sized chunks. Source: I do driver development for devices that DMA at extremely high rates (we're desperately waiting for PCIe 4.0 to become normal—PCIe3.0x16's 126Gb/s is way too slow for us).