3 ms·
Why is something runnable on 4x A10 (see parent sibling) but not 1x4090 in 4-8x the time
by grogenaut 4y ago
Why is something runnable on 4x A10 (see parent sibling) but not 1x4090 in 4-8x the time
- ilyt 4y agoCoz it uses all of the memory so the moment you'd have to spill it to RAM or worse, disk it would be not x4 but x100+ slower
- grogenaut 4y agoSure but that would get you to one inferences around every second. I wait around 1-2 minutes with some online tools for 4 inferences. So seems like it might work fine for a local no data leak version?
- lairv 4y agoThe problem here is memory, so while it should be possible to run inference on a 4090, you would have to constantly move data from VRAM to RAM, though I don't have benchmarks to prove it I think it would be slower than 10x
- gmadsen 4y agodoes that mean it could at least run? even if a magnitude or two slower?
- heisenzombie 4y agoVRAM, you need enough. That’s sort of a binary thing: either you have enough or you don’t. 1x4090 does not.
- grogenaut 4y agoBut at 4x another GPU, they also don't have the vram, right thus 4...?