3 ms·

Yes, ROCm can be used to run frontier models and is being used by OpenAI, Anthropic, and Meta.

• sylware 2mo ago
  I would prefer direct hardware kernel interface. Like linux DMABUFs with userland hardware command ring buffers (I guess this hardware ring buffer instance would be specific to a VMID and a PASID).
  • wmf 2mo ago
    You can do that (tinygrad style) if you want.
    • sylware 2mo ago
      Namely AMD would have to publish the hardware documentation (for instance command packets) of anything specific to inference (unless, this is just their GPU doing only compute).