2 ms·
I'd say it is not the 4GB GPU setup that is the champion of sharding, but the 4xRTX3090 that is significantly cheaper than a single 80GB vram card. Btw, has an
by PeterStuer 3y ago
I'd say it is not the 4GB GPU setup that is the champion of sharding, but the 4xRTX3090 that is significantly cheaper than a single 80GB vram card.
Btw, has anyone experimented with Microsoft's DirectStorage api for this usecase? I understand that the shards will not be very compressable, but maybe the increased IO offloading can have benefits?