Yes, these large language models are extremely constrained by RAM. Traditional PC architecture separate RAM and VRAM, so GPUs are required to have the appropria

Yes, these large language models are extremely constrained by RAM. Traditional PC architecture separate RAM and VRAM, so GPUs are required to have the appropriate amount of memory onboard, in this case achieved via 8 A100s synchronized to act as one compute module. This is why developers such as myself have opted for Apple's new unified architecture to run smaller models locally, as the GPU can access all 64GB. But these small models are still a fraction of the size of GPT-3.5-Turbo and GPT-4.