I’m not sure heterogeneous memory is such a huge deal, due to performance numbers. PCI express is relatively slow in terms of bandwidth, and especially latency.

I’m not sure heterogeneous memory is such a huge deal, due to performance numbers. PCI express is relatively slow in terms of bandwidth, and especially latency. To compensate, GPUs have dedicated piece of hardware to asynchronously copy blocks of data over PCIe. In modern low-level APIs that hardware is even directly exposed to programmers, as transfer queue in Vulkan, and copy command queue in D3D12. Manually moving data with APIs like cudaMemcpy or ID3D11DeviceContext.CopyResource complicates the code, but much faster than unified memory. Especially if you did it correctly with pipelining, and GPU computes something else (like previous batch of work) while the new data is being copied. Speaking of new features, I would rather expect GPGPU users to be interested in DirectStorage technology which allows GPUs to efficiently load data from SSD. That thing is currently Windows-only but supported by all 3 GPU vendors. Because it was implemented primarily for videogames, it works just fine with compute shaders.