What's the point of having the GPU on die for this? Are they expecting people to deploy one of these nodes without dedicated GPUs? It has a ton of NVLink connec

What's the point of having the GPU on die for this? Are they expecting people to deploy one of these nodes without dedicated GPUs? It has a ton of NVLink connections which makes me think that these will often be deployed alongside GPUs which feels weird. The flip side of this is if the GPU can access the main system memory then I could see this being useful for loading big models with much more efficient "offloading" of layers. Even though bandwidth between GPU->LPDDR5 is going to be slow, it's still faster than what traditional PCI-E would allow. The caveat here is that I imagine these machines are $$$ and enterprise only. If something like this was brought to the consumer market though I think it would be very enticing. (If anybody from AMD is reading this, I feel like an architecture like this would be awesome to have. I would love to run Llama 3.1 405b at home and today I see zero path towards doing that for any "reasonable" amount of money (<$10k?).) Edit: It's at the bottom of the article. These are designed to be meshed together via NVLink into one big cluster. Makes sense. I'm really curious how the system RAM would be used in LLM training scenarios, or if these boxes are going to be used for totally different tasks that I have little context into.