3 ms·
I think GP meant zero-copy communication with the GPU, eg. through `newBufferWithBytesNoCopy` [0], which is only possible with unified memory architectures, eg.
by fathyb 3y ago
I think GP meant zero-copy communication with the GPU, eg. through `newBufferWithBytesNoCopy` [0], which is only possible with unified memory architectures, eg. integrated GPUs.
The mmap change was just about mapping the model files in memory instead of copying them, which has less overhead.
[0]: https://developer.apple.com/documentation/metal/mtldevice/1433382-newbufferwithbytesnocopy https://developer.apple.com/documentation/metal/mtldevice/14...
- brucethemoose2 3y agoYeah this is precisely what I meant. I think it is possible in Vulkan too. I brought it up because shuffling weights around takes lots of time, takes up more RAM, and saps memory bandwidth the IGP and CPU desparately need.
- astrange 3y ago> I think GP meant zero-copy communication with the GPU, eg. through `newBufferWithBytesNoCopy` [0], which is only possible with unified memory architectures, eg. integrated GPUs. It can still be beneficial on discrete GPUs because it avoids a copy inside the driver.