If I understand correctly, each chip has 200 MB of RAM, so it takes racks to run a single LLM. That does not sound like progress to me. We need single PCIe bo

If I understand correctly, each chip has 200 MB of RAM, so it takes racks to run a single LLM. That does not sound like progress to me. We need single PCIe boards with dozens or hundreds of GB of RAM and processors that handle it well.