23 ms·
That understanding of the system is correct. To make it practical we've implemented a bunch of optimizations to minimize I/O cost. You can see how it performs o
by bmodel 2y ago
That understanding of the system is correct. To make it practical we've implemented a bunch of optimizations to minimize I/O cost. You can see how it performs on inference with BERT here: https://youtu.be/qsOBFQZtsFM?t=69 https://youtu.be/qsOBFQZtsFM?t=69.
The overheads are larger for training compared to inference, and we are implementing more optimizations to approach native performance.
- radarsat1 2y agoAah ok thanks, that was my basic misunderstanding, my mind just jumped straight to my current training needs but for inference it makes a lot of sense. Thanks for the clarification.
- semitones 2y ago> to approach native performance. The same way one "approaches the sun" when they take the stairs?
- the8472 2y agoI guess there's non-negligible optimization potential, e.g. by doing hash-based caching. If the same data gets uploaded twice they can have the blob already sitting somewhere closer to the machine.
- ruined 2y agoyes. my building has stairs and i find them useful because usually i don't need to go to the sun
- ranger_danger 2y agoIs DirectX support possible any time soon? This would be huge for Windows VMs on Linux...
- zozbot234 2y agoYou could use unofficial Windows drivers for virtio-gpu, that are specifically intended for VM use.
- ranger_danger 2y agoIndeed, it's just that it's not very feature-complete or stable yet.