4 ms·
After digging it looks like 70B Q4 requires 35GB or so. So yes the AMD comparison is a strix halo not swapping vs a 4090 that is swapping. Seems kinda mislead
by sliken 2y ago
After digging it looks like 70B Q4 requires 35GB or so. So yes the AMD comparison is a strix halo not swapping vs a 4090 that is swapping. Seems kinda misleading, but is a real advantage for larger models.