> Do models that require multiple cards to run in memory run with equivalent performance of the multiple cards or are they throttled by the memory access? It d

> Do models that require multiple cards to run in memory run with equivalent performance of the multiple cards or are they throttled by the memory access? It depends, but some frameworks just run the layers sequentially. So you get ~50% utilization from 2 GPUs unless you pipeline requests from multiple users