4 ms·
The NPU in previous AMD generation has unfortunately it's own dedicated RAM that is far too small for an LLM. But even those should comfortably run an SLM of a
by throwa356262 2mo ago
The NPU in previous AMD generation has unfortunately it's own dedicated RAM that is far too small for an LLM.
But even those should comfortably run an SLM of around 100-400M parameters at crazy high speeds and with minimal power usage.
- rbanffy 2mo ago> an SLM of around 100-400M parameters at crazy high speeds and with minimal power usage Software support for that kind of usage, when available, will be immensely impactful.