7 ms·
The problem here is memory, so while it should be possible to run inference on a 4090, you would have to constantly move data from VRAM to RAM, though I don't h
by lairv 4y ago
The problem here is memory, so while it should be possible to run inference on a 4090, you would have to constantly move data from VRAM to RAM, though I don't have benchmarks to prove it I think it would be slower than 10x
- gmadsen 4y agodoes that mean it could at least run? even if a magnitude or two slower?