3 ms·

One second, don't LLMs generally run in VRAM? If you put them in regular RAM, don't they have to go through the CPU which kills performance?

• pxeger1 1y ago
  The mentioned CPU uses unified memory for its built in GPU / NPU. I.e. some portion of what could ordinarily be system RAM is given to the GPU instead of the CPU
  • oblio 1y ago
    Ah, now I see, didn't know that was feasible in the PC world. Glad that it's becoming an option.